[==========] Running 56 tests from 4 test suites.
[----------] Global test environment set-up.
[----------] 30 tests from SecurityITest
[ RUN ] SecurityITest.TestAuthorizationOnListTablets
Loading random data
Initializing database '/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786416707705494-12843-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 02:51:47 dist-test-slave-9gcw krb5kdc[12853](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 02:51:47 dist-test-slave-9gcw krb5kdc[12853](info): set up 2 sockets
Aug 11 02:51:47 dist-test-slave-9gcw krb5kdc[12853](info): commencing operation
krb5kdc: starting...
WARNING: Logging before InitGoogleLogging() is written to STDERR
W20260811 02:51:48.883597 12843 mini_kdc.cc:121] Time spent starting KDC: real 1.137s user 0.007s sys 0.002s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786416707705494-12843-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786416707705494-12843-0/krb5kdc/test-user.keytab.
Aug 11 02:51:48 dist-test-slave-9gcw krb5kdc[12853](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416708, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T02:51:48Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T02:51:48Z Disabled control of system clock
WARNING: no policy specified for kudu/127.12.138.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.138.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.138.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786416707705494-12843-0/krb5kdc/kudu_127.12.138.254.keytab.
Entry for principal kudu/127.12.138.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786416707705494-12843-0/krb5kdc/kudu_127.12.138.254.keytab.
I20260811 02:51:49.037525 12843 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDawEwS/build/debug/bin/kudu
/tmp/dist-test-taskDawEwS/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786416707705494-12843-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786416707705494-12843-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786416707705494-12843-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786416707705494-12843-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.138.254:37061
--webserver_interface=127.12.138.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786416707705494-12843-0/krb5kdc/kudu_127.12.138.254.keytab
--principal=kudu/127.12.138.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.138.212:41343
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.12.138.254:37061
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786416707705494-12843-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786416707705494-12843-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786416707705494-12843-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 02:51:49.153457 12870 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 02:51:49.153765 12870 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 02:51:49.154021 12870 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 02:51:49.158684 12870 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 02:51:49.158790 12870 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 02:51:49.158830 12870 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 02:51:49.158869 12870 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 02:51:49.158900 12870 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 02:51:49.164260 12870 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.138.212:41343
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786416707705494-12843-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786416707705494-12843-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.12.138.254:37061
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786416707705494-12843-0/krb5kdc/kudu_127.12.138.254.keytab
--principal=kudu/127.12.138.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.138.254:37061
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786416707705494-12843-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.12.138.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12870
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786416707705494-12843-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 02:09:49 UTC on ccycloud.quasar-fsnqqc.root.comops.site
build id 4579570
I20260811 02:51:49.165397 12870 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 02:51:49.166446 12870 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 02:51:49.175431 12877 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:51:49.175530 12880 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:51:49.175455 12878 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:51:49.175763 12870 server_base.cc:1034] running on GCE node
I20260811 02:51:49.176385 12870 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 02:51:49.177596 12870 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 02:51:49.178802 12870 hybrid_clock.cc:648] HybridClock initialized: now 1786416709178823 us; error 70 us; skew 500 ppm
Aug 11 02:51:49 dist-test-slave-9gcw krb5kdc[12853](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416709, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.138.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 02:51:49.181900 12870 init.cc:377] Logged in from keytab as kudu/127.12.138.254@KRBTEST.COM (short username kudu)
I20260811 02:51:49.182884 12870 webserver.cc:460] Webserver started at http://127.12.138.254:33023/ using document root <none> and password file <none>
I20260811 02:51:49.183463 12870 fs_manager.cc:359] Metadata directory not provided
I20260811 02:51:49.183528 12870 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 02:51:49.183763 12870 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 02:51:49.186064 12870 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786416707705494-12843-0/minicluster-data/master-0/data/instance:
uuid: "83da7932d3c84a6a9e737fe8a6a48b7b"
format_stamp: "Formatted at 2026-08-11 02:51:49 on dist-test-slave-9gcw"
server_key: "6d7c83c2429d3be32268f17c14e059d4"
server_key_iv: "47995a24aac38a72baef67085041fbc2"
server_key_version: "encryptionkey@0"
I20260811 02:51:49.186659 12870 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786416707705494-12843-0/minicluster-data/master-0/wal/instance:
uuid: "83da7932d3c84a6a9e737fe8a6a48b7b"
format_stamp: "Formatted at 2026-08-11 02:51:49 on dist-test-slave-9gcw"
server_key: "6d7c83c2429d3be32268f17c14e059d4"
server_key_iv: "47995a24aac38a72baef67085041fbc2"
server_key_version: "encryptionkey@0"
I20260811 02:51:49.190562 12870 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260811 02:51:49.193434 12889 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 02:51:49.194789 12870 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260811 02:51:49.194933 12870 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786416707705494-12843-0/minicluster-data/master-0/data,/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786416707705494-12843-0/minicluster-data/master-0/wal
uuid: "83da7932d3c84a6a9e737fe8a6a48b7b"
format_stamp: "Formatted at 2026-08-11 02:51:49 on dist-test-slave-9gcw"
server_key: "6d7c83c2429d3be32268f17c14e059d4"
server_key_iv: "47995a24aac38a72baef67085041fbc2"
server_key_version: "encryptionkey@0"
I20260811 02:51:49.195048 12870 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786416707705494-12843-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786416707705494-12843-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786416707705494-12843-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 02:51:49.211889 12870 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 02:51:49.215200 12870 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 02:51:49.215432 12870 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 02:51:49.222822 12870 rpc_server.cc:307] RPC server started. Bound to: 127.12.138.254:37061
I20260811 02:51:49.222880 12953 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.138.254:37061 every 8 connection(s)
I20260811 02:51:49.223829 12870 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786416707705494-12843-0/minicluster-data/master-0/data/info.pb
I20260811 02:51:49.225863 12843 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDawEwS/build/debug/bin/kudu as pid 12870
I20260811 02:51:49.226063 12843 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786416707705494-12843-0/minicluster-data/master-0/wal/instance
I20260811 02:51:49.227380 12954 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 02:51:49.227787 12843 external_mini_cluster.cc:1428] Setting key 4756a9e868b711c90842db563eca73fe
I20260811 02:51:49.234705 12954 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 83da7932d3c84a6a9e737fe8a6a48b7b: Bootstrap starting.
I20260811 02:51:49.237552 12954 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 83da7932d3c84a6a9e737fe8a6a48b7b: Neither blocks nor log segments found. Creating new log.
I20260811 02:51:49.238409 12954 log.cc:824] T 00000000000000000000000000000000 P 83da7932d3c84a6a9e737fe8a6a48b7b: Log is configured to *not* fsync() on all Append() calls
I20260811 02:51:49.240762 12954 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 83da7932d3c84a6a9e737fe8a6a48b7b: No bootstrap required, opened a new log
I20260811 02:51:49.245222 12954 raft_consensus.cc:348] T 00000000000000000000000000000000 P 83da7932d3c84a6a9e737fe8a6a48b7b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "83da7932d3c84a6a9e737fe8a6a48b7b" member_type: VOTER last_known_addr { host: "127.12.138.254" port: 37061 } }
I20260811 02:51:49.245513 12954 raft_consensus.cc:374] T 00000000000000000000000000000000 P 83da7932d3c84a6a9e737fe8a6a48b7b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 02:51:49.245587 12954 raft_consensus.cc:729] T 00000000000000000000000000000000 P 83da7932d3c84a6a9e737fe8a6a48b7b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 83da7932d3c84a6a9e737fe8a6a48b7b, State: Initialized, Role: FOLLOWER
I20260811 02:51:49.246074 12954 consensus_queue.cc:260] T 00000000000000000000000000000000 P 83da7932d3c84a6a9e737fe8a6a48b7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "83da7932d3c84a6a9e737fe8a6a48b7b" member_type: VOTER last_known_addr { host: "127.12.138.254" port: 37061 } }
I20260811 02:51:49.246279 12954 raft_consensus.cc:388] T 00000000000000000000000000000000 P 83da7932d3c84a6a9e737fe8a6a48b7b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 02:51:49.246354 12954 raft_consensus.cc:482] T 00000000000000000000000000000000 P 83da7932d3c84a6a9e737fe8a6a48b7b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 02:51:49.246460 12954 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 83da7932d3c84a6a9e737fe8a6a48b7b [term 0 FOLLOWER]: Advancing to term 1
I20260811 02:51:49.247757 12954 raft_consensus.cc:504] T 00000000000000000000000000000000 P 83da7932d3c84a6a9e737fe8a6a48b7b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "83da7932d3c84a6a9e737fe8a6a48b7b" member_type: VOTER last_known_addr { host: "127.12.138.254" port: 37061 } }
I20260811 02:51:49.248147 12954 leader_election.cc:302] T 00000000000000000000000000000000 P 83da7932d3c84a6a9e737fe8a6a48b7b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 83da7932d3c84a6a9e737fe8a6a48b7b; no voters:
I20260811 02:51:49.248469 12954 leader_election.cc:288] T 00000000000000000000000000000000 P 83da7932d3c84a6a9e737fe8a6a48b7b [CANDIDATE]: Term 1 election: Requested vote from peers
Aug 11 02:51:49 dist-test-slave-9gcw krb5kdc[12853](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416708, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.12.138.254@KRBTEST.COM
I20260811 02:51:49.248600 12961 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 83da7932d3c84a6a9e737fe8a6a48b7b [term 1 FOLLOWER]: Leader election won for term 1
I20260811 02:51:49.248880 12961 raft_consensus.cc:686] T 00000000000000000000000000000000 P 83da7932d3c84a6a9e737fe8a6a48b7b [term 1 LEADER]: Becoming Leader. State: Replica: 83da7932d3c84a6a9e737fe8a6a48b7b, State: Running, Role: LEADER
I20260811 02:51:49.249305 12961 consensus_queue.cc:237] T 00000000000000000000000000000000 P 83da7932d3c84a6a9e737fe8a6a48b7b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "83da7932d3c84a6a9e737fe8a6a48b7b" member_type: VOTER last_known_addr { host: "127.12.138.254" port: 37061 } }
I20260811 02:51:49.249867 12954 sys_catalog.cc:564] T 00000000000000000000000000000000 P 83da7932d3c84a6a9e737fe8a6a48b7b [sys.catalog]: configured and running, proceeding with master startup.
I20260811 02:51:49.251690 12965 sys_catalog.cc:455] T 00000000000000000000000000000000 P 83da7932d3c84a6a9e737fe8a6a48b7b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 83da7932d3c84a6a9e737fe8a6a48b7b. Latest consensus state: current_term: 1 leader_uuid: "83da7932d3c84a6a9e737fe8a6a48b7b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "83da7932d3c84a6a9e737fe8a6a48b7b" member_type: VOTER last_known_addr { host: "127.12.138.254" port: 37061 } } }
I20260811 02:51:49.251840 12965 sys_catalog.cc:458] T 00000000000000000000000000000000 P 83da7932d3c84a6a9e737fe8a6a48b7b [sys.catalog]: This master's current role is: LEADER
I20260811 02:51:49.252158 12964 sys_catalog.cc:455] T 00000000000000000000000000000000 P 83da7932d3c84a6a9e737fe8a6a48b7b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "83da7932d3c84a6a9e737fe8a6a48b7b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "83da7932d3c84a6a9e737fe8a6a48b7b" member_type: VOTER last_known_addr { host: "127.12.138.254" port: 37061 } } }
I20260811 02:51:49.252562 12981 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 02:51:49.252768 12964 sys_catalog.cc:458] T 00000000000000000000000000000000 P 83da7932d3c84a6a9e737fe8a6a48b7b [sys.catalog]: This master's current role is: LEADER
I20260811 02:51:49.255976 12981 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 02:51:49.256271 12957 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:51:49.231711 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59668 (local address 127.12.138.254:37061)
0811 02:51:49.232202 (+ 491us) server_negotiation.cc:206] Beginning negotiation
0811 02:51:49.232221 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0811 02:51:49.233268 (+ 1047us) server_negotiation.cc:407] Connection header received
0811 02:51:49.237496 (+ 4228us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:51:49.237513 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:51:49.237907 (+ 394us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:51:49.238167 (+ 260us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:51:49.241062 (+ 2895us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:51:49.242135 (+ 1073us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:51:49.243954 (+ 1819us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:51:49.244407 (+ 453us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:51:49.244524 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:51:49.249648 (+ 5124us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:51:49.249654 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:51:49.249668 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:51:49.249700 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:51:49.252116 (+ 2416us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:51:49.252909 (+ 793us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:51:49.252917 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:51:49.252923 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:51:49.253082 (+ 159us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:51:49.253577 (+ 495us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:51:49.253580 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:51:49.253582 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:51:49.254083 (+ 501us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:51:49.254290 (+ 207us) server_negotiation.cc:1035] Waiting for connection context
0811 02:51:49.255017 (+ 727us) server_negotiation.cc:299] Negotiation successful
0811 02:51:49.255326 (+ 309us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":324,"thread_start_us":136,"threads_started":1}
I20260811 02:51:49.261914 12981 catalog_manager.cc:1347] Generated new cluster ID: b61ab213fed8437ba93f1b62edc8092c
I20260811 02:51:49.261996 12981 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 02:51:49.275959 12981 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 02:51:49.276973 12981 catalog_manager.cc:1504] Loading token signing keys...
I20260811 02:51:49.284987 12981 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 83da7932d3c84a6a9e737fe8a6a48b7b: Generated new TSK 0
I20260811 02:51:49.285882 12981 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.12.138.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.138.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.138.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786416707705494-12843-0/krb5kdc/kudu_127.12.138.193.keytab.
Entry for principal kudu/127.12.138.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786416707705494-12843-0/krb5kdc/kudu_127.12.138.193.keytab.
I20260811 02:51:49.321302 12843 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDawEwS/build/debug/bin/kudu
/tmp/dist-test-taskDawEwS/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786416707705494-12843-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786416707705494-12843-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786416707705494-12843-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786416707705494-12843-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.138.193:0
--local_ip_for_outbound_sockets=127.12.138.193
--webserver_interface=127.12.138.193
--webserver_port=0
--tserver_master_addrs=127.12.138.254:37061
--keytab_file=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786416707705494-12843-0/krb5kdc/kudu_127.12.138.193.keytab
--principal=kudu/127.12.138.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.138.212:41343
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786416707705494-12843-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786416707705494-12843-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786416707705494-12843-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 02:51:49.440394 12989 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 02:51:49.440721 12989 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 02:51:49.440801 12989 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 02:51:49.444840 12989 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 02:51:49.444952 12989 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 02:51:49.445052 12989 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.138.193
I20260811 02:51:49.450757 12989 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.138.212:41343
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786416707705494-12843-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786416707705494-12843-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786416707705494-12843-0/krb5kdc/kudu_127.12.138.193.keytab
--principal=kudu/127.12.138.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.138.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786416707705494-12843-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.12.138.193
--webserver_port=0
--tserver_master_addrs=127.12.138.254:37061
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12989
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.138.193
--log_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786416707705494-12843-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 02:09:49 UTC on ccycloud.quasar-fsnqqc.root.comops.site
build id 4579570
I20260811 02:51:49.451969 12989 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 02:51:49.453068 12989 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 02:51:49.463120 12994 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:51:49.463150 12997 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:51:49.463120 12995 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:51:49.463411 12989 server_base.cc:1034] running on GCE node
I20260811 02:51:49.463913 12989 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 02:51:49.464514 12989 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 02:51:49.465729 12989 hybrid_clock.cc:648] HybridClock initialized: now 1786416709465695 us; error 46 us; skew 500 ppm
Aug 11 02:51:49 dist-test-slave-9gcw krb5kdc[12853](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416709, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.138.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 02:51:49.468585 12989 init.cc:377] Logged in from keytab as kudu/127.12.138.193@KRBTEST.COM (short username kudu)
I20260811 02:51:49.469676 12989 webserver.cc:460] Webserver started at http://127.12.138.193:39231/ using document root <none> and password file <none>
I20260811 02:51:49.470315 12989 fs_manager.cc:359] Metadata directory not provided
I20260811 02:51:49.470424 12989 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 02:51:49.470685 12989 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 02:51:49.472836 12989 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786416707705494-12843-0/minicluster-data/ts-0/data/instance:
uuid: "66b483a261db4467b736d300023c09c4"
format_stamp: "Formatted at 2026-08-11 02:51:49 on dist-test-slave-9gcw"
server_key: "2d0d41a4b506c0c1a99dce06bd1d8e51"
server_key_iv: "3dfdafa226b878d875cb71a370522c16"
server_key_version: "encryptionkey@0"
I20260811 02:51:49.473466 12989 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786416707705494-12843-0/minicluster-data/ts-0/wal/instance:
uuid: "66b483a261db4467b736d300023c09c4"
format_stamp: "Formatted at 2026-08-11 02:51:49 on dist-test-slave-9gcw"
server_key: "2d0d41a4b506c0c1a99dce06bd1d8e51"
server_key_iv: "3dfdafa226b878d875cb71a370522c16"
server_key_version: "encryptionkey@0"
I20260811 02:51:49.477447 12989 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260811 02:51:49.480284 13005 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 02:51:49.481604 12989 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 02:51:49.481812 12989 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786416707705494-12843-0/minicluster-data/ts-0/data,/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786416707705494-12843-0/minicluster-data/ts-0/wal
uuid: "66b483a261db4467b736d300023c09c4"
format_stamp: "Formatted at 2026-08-11 02:51:49 on dist-test-slave-9gcw"
server_key: "2d0d41a4b506c0c1a99dce06bd1d8e51"
server_key_iv: "3dfdafa226b878d875cb71a370522c16"
server_key_version: "encryptionkey@0"
I20260811 02:51:49.481940 12989 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786416707705494-12843-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786416707705494-12843-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786416707705494-12843-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 02:51:49.506769 12989 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 02:51:49.510291 12989 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 02:51:49.510519 12989 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 02:51:49.511144 12989 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 02:51:49.512027 12989 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 02:51:49.512099 12989 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 02:51:49.512168 12989 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 02:51:49.512221 12989 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 02:51:49.521894 12989 rpc_server.cc:307] RPC server started. Bound to: 127.12.138.193:37807
I20260811 02:51:49.521926 13139 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.138.193:37807 every 8 connection(s)
I20260811 02:51:49.522812 12989 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786416707705494-12843-0/minicluster-data/ts-0/data/info.pb
I20260811 02:51:49.529405 12843 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDawEwS/build/debug/bin/kudu as pid 12989
I20260811 02:51:49.529525 12843 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786416707705494-12843-0/minicluster-data/ts-0/wal/instance
I20260811 02:51:49.529801 12843 external_mini_cluster.cc:1428] Setting key 07276b8e9f2ceaeb83b7e42c9737a47b
Aug 11 02:51:49 dist-test-slave-9gcw krb5kdc[12853](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416709, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.138.193@KRBTEST.COM for kudu/127.12.138.254@KRBTEST.COM
I20260811 02:51:49.537515 12957 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:51:49.524645 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.138.193:53811 (local address 127.12.138.254:37061)
0811 02:51:49.524959 (+ 314us) server_negotiation.cc:206] Beginning negotiation
0811 02:51:49.524963 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 02:51:49.525651 (+ 688us) server_negotiation.cc:407] Connection header received
0811 02:51:49.527107 (+ 1456us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:51:49.527111 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:51:49.527204 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:51:49.527312 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:51:49.528580 (+ 1268us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:51:49.529133 (+ 553us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:51:49.530501 (+ 1368us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:51:49.530801 (+ 300us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:51:49.530898 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:51:49.533466 (+ 2568us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:51:49.533475 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:51:49.533477 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:51:49.533500 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:51:49.534915 (+ 1415us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:51:49.535571 (+ 656us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:51:49.535573 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:51:49.535575 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:51:49.535740 (+ 165us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:51:49.536187 (+ 447us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:51:49.536190 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:51:49.536191 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:51:49.536392 (+ 201us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:51:49.536483 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0811 02:51:49.537274 (+ 791us) server_negotiation.cc:299] Negotiation successful
0811 02:51:49.537391 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260811 02:51:49.538769 13142 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:51:49.525009 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.138.254:37061 (local address 127.12.138.193:53811)
0811 02:51:49.525486 (+ 477us) negotiation.cc:107] Waiting for socket to connect
0811 02:51:49.525531 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0811 02:51:49.526885 (+ 1354us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:51:49.527503 (+ 618us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:51:49.527510 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:51:49.527889 (+ 379us) client_negotiation.cc:189] Negotiated authn=SASL
0811 02:51:49.528427 (+ 538us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:51:49.528441 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:51:49.529314 (+ 873us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:51:49.529322 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:51:49.530368 (+ 1046us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:51:49.530378 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:51:49.530961 (+ 583us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:51:49.530965 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:51:49.531189 (+ 224us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:51:49.531796 (+ 607us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 02:51:49.531813 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 02:51:49.533300 (+ 1487us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 02:51:49.535036 (+ 1736us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:51:49.535045 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:51:49.535063 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:51:49.535450 (+ 387us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:51:49.535849 (+ 399us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:51:49.535854 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:51:49.535859 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:51:49.536049 (+ 190us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:51:49.536512 (+ 463us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 02:51:49.536522 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 02:51:49.536928 (+ 406us) client_negotiation.cc:769] Sending connection context
0811 02:51:49.537192 (+ 264us) client_negotiation.cc:240] Negotiation successful
0811 02:51:49.537569 (+ 377us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":287,"thread_start_us":143,"threads_started":1}
I20260811 02:51:49.540148 13140 heartbeater.cc:347] Connected to a master server at 127.12.138.254:37061
I20260811 02:51:49.540403 13140 heartbeater.cc:464] Registering TS with master...
I20260811 02:51:49.540925 13140 heartbeater.cc:511] Master 127.12.138.254:37061 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.12.138.194@KRBTEST.COM; defaulting to no policy
I20260811 02:51:49.542562 12910 ts_manager.cc:194] Registered new tserver with Master: 66b483a261db4467b736d300023c09c4 (127.12.138.193:37807)
I20260811 02:51:49.543818 12910 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.12.138.193@KRBTEST.COM'} at 127.12.138.193:53811
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.138.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.138.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786416707705494-12843-0/krb5kdc/kudu_127.12.138.194.keytab.
Entry for principal kudu/127.12.138.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786416707705494-12843-0/krb5kdc/kudu_127.12.138.194.keytab.
I20260811 02:51:49.562390 12843 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDawEwS/build/debug/bin/kudu
/tmp/dist-test-taskDawEwS/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786416707705494-12843-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786416707705494-12843-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786416707705494-12843-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786416707705494-12843-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.138.194:0
--local_ip_for_outbound_sockets=127.12.138.194
--webserver_interface=127.12.138.194
--webserver_port=0
--tserver_master_addrs=127.12.138.254:37061
--keytab_file=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786416707705494-12843-0/krb5kdc/kudu_127.12.138.194.keytab
--principal=kudu/127.12.138.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.138.212:41343
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786416707705494-12843-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786416707705494-12843-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786416707705494-12843-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 02:51:49.680979 13146 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 02:51:49.681381 13146 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 02:51:49.681469 13146 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 02:51:49.686308 13146 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 02:51:49.686393 13146 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 02:51:49.686501 13146 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.138.194
I20260811 02:51:49.692492 13146 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.138.212:41343
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786416707705494-12843-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786416707705494-12843-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786416707705494-12843-0/krb5kdc/kudu_127.12.138.194.keytab
--principal=kudu/127.12.138.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.138.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786416707705494-12843-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.12.138.194
--webserver_port=0
--tserver_master_addrs=127.12.138.254:37061
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13146
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.138.194
--log_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786416707705494-12843-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 02:09:49 UTC on ccycloud.quasar-fsnqqc.root.comops.site
build id 4579570
I20260811 02:51:49.693819 13146 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 02:51:49.694891 13146 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 02:51:49.705221 13152 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:51:49.705222 13153 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:51:49.705241 13156 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:51:49.705657 13146 server_base.cc:1034] running on GCE node
I20260811 02:51:49.706066 13146 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 02:51:49.706758 13146 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 02:51:49.707978 13146 hybrid_clock.cc:648] HybridClock initialized: now 1786416709707959 us; error 29 us; skew 500 ppm
Aug 11 02:51:49 dist-test-slave-9gcw krb5kdc[12853](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416709, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.138.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 02:51:49.710952 13146 init.cc:377] Logged in from keytab as kudu/127.12.138.194@KRBTEST.COM (short username kudu)
I20260811 02:51:49.711902 13146 webserver.cc:460] Webserver started at http://127.12.138.194:33407/ using document root <none> and password file <none>
I20260811 02:51:49.712481 13146 fs_manager.cc:359] Metadata directory not provided
I20260811 02:51:49.712572 13146 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 02:51:49.712831 13146 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 02:51:49.714994 13146 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786416707705494-12843-0/minicluster-data/ts-1/data/instance:
uuid: "3dc4a7b1838343c190a157c21cc7031e"
format_stamp: "Formatted at 2026-08-11 02:51:49 on dist-test-slave-9gcw"
server_key: "dd3bc2d2c62046fb0e972ab8cae86e20"
server_key_iv: "810d5f3a225fabb471c17a9de46f06b0"
server_key_version: "encryptionkey@0"
I20260811 02:51:49.715606 13146 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786416707705494-12843-0/minicluster-data/ts-1/wal/instance:
uuid: "3dc4a7b1838343c190a157c21cc7031e"
format_stamp: "Formatted at 2026-08-11 02:51:49 on dist-test-slave-9gcw"
server_key: "dd3bc2d2c62046fb0e972ab8cae86e20"
server_key_iv: "810d5f3a225fabb471c17a9de46f06b0"
server_key_version: "encryptionkey@0"
I20260811 02:51:49.719574 13146 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.001s sys 0.004s
I20260811 02:51:49.722206 13165 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 02:51:49.723512 13146 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260811 02:51:49.723655 13146 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786416707705494-12843-0/minicluster-data/ts-1/data,/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786416707705494-12843-0/minicluster-data/ts-1/wal
uuid: "3dc4a7b1838343c190a157c21cc7031e"
format_stamp: "Formatted at 2026-08-11 02:51:49 on dist-test-slave-9gcw"
server_key: "dd3bc2d2c62046fb0e972ab8cae86e20"
server_key_iv: "810d5f3a225fabb471c17a9de46f06b0"
server_key_version: "encryptionkey@0"
I20260811 02:51:49.723780 13146 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786416707705494-12843-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786416707705494-12843-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786416707705494-12843-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 02:51:49.744254 13146 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 02:51:49.747733 13146 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 02:51:49.747961 13146 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 02:51:49.748593 13146 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 02:51:49.749575 13146 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 02:51:49.749653 13146 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 02:51:49.749830 13146 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 02:51:49.749864 13146 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 02:51:49.759956 13146 rpc_server.cc:307] RPC server started. Bound to: 127.12.138.194:35223
I20260811 02:51:49.759980 13291 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.138.194:35223 every 8 connection(s)
I20260811 02:51:49.760949 13146 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786416707705494-12843-0/minicluster-data/ts-1/data/info.pb
I20260811 02:51:49.770105 12843 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDawEwS/build/debug/bin/kudu as pid 13146
I20260811 02:51:49.770202 12843 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786416707705494-12843-0/minicluster-data/ts-1/wal/instance
I20260811 02:51:49.770530 12843 external_mini_cluster.cc:1428] Setting key f711e8f8ec0a6cd124bd0092e0c2440a
Aug 11 02:51:49 dist-test-slave-9gcw krb5kdc[12853](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416709, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.138.194@KRBTEST.COM for kudu/127.12.138.254@KRBTEST.COM
I20260811 02:51:49.775583 12957 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:51:49.762927 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.138.194:42111 (local address 127.12.138.254:37061)
0811 02:51:49.763108 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0811 02:51:49.763113 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 02:51:49.764061 (+ 948us) server_negotiation.cc:407] Connection header received
0811 02:51:49.765436 (+ 1375us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:51:49.765439 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:51:49.765489 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:51:49.765568 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:51:49.766780 (+ 1212us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:51:49.767348 (+ 568us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:51:49.768477 (+ 1129us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:51:49.768693 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:51:49.768772 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:51:49.771621 (+ 2849us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:51:49.771625 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:51:49.771627 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:51:49.771652 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:51:49.773213 (+ 1561us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:51:49.773898 (+ 685us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:51:49.773901 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:51:49.773903 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:51:49.773985 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:51:49.774412 (+ 427us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:51:49.774416 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:51:49.774418 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:51:49.774621 (+ 203us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:51:49.774733 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0811 02:51:49.775265 (+ 532us) server_negotiation.cc:299] Negotiation successful
0811 02:51:49.775416 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260811 02:51:49.776434 13295 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:51:49.763396 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.138.254:37061 (local address 127.12.138.194:42111)
0811 02:51:49.763873 (+ 477us) negotiation.cc:107] Waiting for socket to connect
0811 02:51:49.763916 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0811 02:51:49.765204 (+ 1288us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:51:49.765734 (+ 530us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:51:49.765742 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:51:49.766117 (+ 375us) client_negotiation.cc:189] Negotiated authn=SASL
0811 02:51:49.766636 (+ 519us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:51:49.766649 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:51:49.767465 (+ 816us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:51:49.767467 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:51:49.768370 (+ 903us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:51:49.768379 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:51:49.768838 (+ 459us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:51:49.768841 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:51:49.768986 (+ 145us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:51:49.769685 (+ 699us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 02:51:49.769702 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 02:51:49.771437 (+ 1735us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 02:51:49.773347 (+ 1910us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:51:49.773356 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:51:49.773376 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:51:49.773782 (+ 406us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:51:49.774072 (+ 290us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:51:49.774078 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:51:49.774083 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:51:49.774310 (+ 227us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:51:49.774732 (+ 422us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 02:51:49.774741 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 02:51:49.775011 (+ 270us) client_negotiation.cc:769] Sending connection context
0811 02:51:49.775220 (+ 209us) client_negotiation.cc:240] Negotiation successful
0811 02:51:49.775601 (+ 381us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":279,"thread_start_us":128,"threads_started":1}
I20260811 02:51:49.777659 13292 heartbeater.cc:347] Connected to a master server at 127.12.138.254:37061
I20260811 02:51:49.777951 13292 heartbeater.cc:464] Registering TS with master...
I20260811 02:51:49.778623 13292 heartbeater.cc:511] Master 127.12.138.254:37061 requested a full tablet report, sending...
I20260811 02:51:49.779934 12910 ts_manager.cc:194] Registered new tserver with Master: 3dc4a7b1838343c190a157c21cc7031e (127.12.138.194:35223)
I20260811 02:51:49.780795 12910 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.12.138.194@KRBTEST.COM'} at 127.12.138.194:42111
WARNING: no policy specified for kudu/127.12.138.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.138.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.138.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786416707705494-12843-0/krb5kdc/kudu_127.12.138.195.keytab.
Entry for principal kudu/127.12.138.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786416707705494-12843-0/krb5kdc/kudu_127.12.138.195.keytab.
I20260811 02:51:49.802507 12843 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDawEwS/build/debug/bin/kudu
/tmp/dist-test-taskDawEwS/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786416707705494-12843-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786416707705494-12843-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786416707705494-12843-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786416707705494-12843-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.138.195:0
--local_ip_for_outbound_sockets=127.12.138.195
--webserver_interface=127.12.138.195
--webserver_port=0
--tserver_master_addrs=127.12.138.254:37061
--keytab_file=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786416707705494-12843-0/krb5kdc/kudu_127.12.138.195.keytab
--principal=kudu/127.12.138.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.138.212:41343
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786416707705494-12843-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786416707705494-12843-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786416707705494-12843-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 02:51:49.918134 13298 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 02:51:49.918390 13298 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 02:51:49.918434 13298 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 02:51:49.922377 13298 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 02:51:49.922438 13298 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 02:51:49.922497 13298 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.138.195
I20260811 02:51:49.927953 13298 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.138.212:41343
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786416707705494-12843-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786416707705494-12843-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786416707705494-12843-0/krb5kdc/kudu_127.12.138.195.keytab
--principal=kudu/127.12.138.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.138.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786416707705494-12843-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.12.138.195
--webserver_port=0
--tserver_master_addrs=127.12.138.254:37061
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13298
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.138.195
--log_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786416707705494-12843-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 02:09:49 UTC on ccycloud.quasar-fsnqqc.root.comops.site
build id 4579570
I20260811 02:51:49.929100 13298 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 02:51:49.930181 13298 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 02:51:49.940186 13305 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:51:49.940200 13304 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:51:49.940200 13307 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:51:49.940455 13298 server_base.cc:1034] running on GCE node
I20260811 02:51:49.940949 13298 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 02:51:49.941640 13298 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 02:51:49.942802 13298 hybrid_clock.cc:648] HybridClock initialized: now 1786416709942772 us; error 44 us; skew 500 ppm
Aug 11 02:51:49 dist-test-slave-9gcw krb5kdc[12853](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416709, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.138.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 02:51:49.945624 13298 init.cc:377] Logged in from keytab as kudu/127.12.138.195@KRBTEST.COM (short username kudu)
I20260811 02:51:49.946476 13298 webserver.cc:460] Webserver started at http://127.12.138.195:45741/ using document root <none> and password file <none>
I20260811 02:51:49.947010 13298 fs_manager.cc:359] Metadata directory not provided
I20260811 02:51:49.947068 13298 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 02:51:49.947235 13298 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 02:51:49.949314 13298 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786416707705494-12843-0/minicluster-data/ts-2/data/instance:
uuid: "a043f3ac80984b90a7bc97ac6bfcf9af"
format_stamp: "Formatted at 2026-08-11 02:51:49 on dist-test-slave-9gcw"
server_key: "b7b2ce6f8a4036fd5903373ecc720e79"
server_key_iv: "bcce487696ad2c20631fad154a5e7351"
server_key_version: "encryptionkey@0"
I20260811 02:51:49.949867 13298 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786416707705494-12843-0/minicluster-data/ts-2/wal/instance:
uuid: "a043f3ac80984b90a7bc97ac6bfcf9af"
format_stamp: "Formatted at 2026-08-11 02:51:49 on dist-test-slave-9gcw"
server_key: "b7b2ce6f8a4036fd5903373ecc720e79"
server_key_iv: "bcce487696ad2c20631fad154a5e7351"
server_key_version: "encryptionkey@0"
I20260811 02:51:49.953881 13298 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260811 02:51:49.956425 13316 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 02:51:49.957839 13298 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260811 02:51:49.957937 13298 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786416707705494-12843-0/minicluster-data/ts-2/data,/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786416707705494-12843-0/minicluster-data/ts-2/wal
uuid: "a043f3ac80984b90a7bc97ac6bfcf9af"
format_stamp: "Formatted at 2026-08-11 02:51:49 on dist-test-slave-9gcw"
server_key: "b7b2ce6f8a4036fd5903373ecc720e79"
server_key_iv: "bcce487696ad2c20631fad154a5e7351"
server_key_version: "encryptionkey@0"
I20260811 02:51:49.958078 13298 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786416707705494-12843-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786416707705494-12843-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786416707705494-12843-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 02:51:49.981194 13298 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 02:51:49.984377 13298 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 02:51:49.984593 13298 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 02:51:49.985280 13298 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 02:51:49.986215 13298 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 02:51:49.986269 13298 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 02:51:49.986310 13298 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 02:51:49.986363 13298 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 02:51:49.996007 13298 rpc_server.cc:307] RPC server started. Bound to: 127.12.138.195:36069
I20260811 02:51:49.996102 13449 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.138.195:36069 every 8 connection(s)
I20260811 02:51:49.997007 13298 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786416707705494-12843-0/minicluster-data/ts-2/data/info.pb
I20260811 02:51:50.000434 12843 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDawEwS/build/debug/bin/kudu as pid 13298
I20260811 02:51:50.000548 12843 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786416707705494-12843-0/minicluster-data/ts-2/wal/instance
I20260811 02:51:50.000844 12843 external_mini_cluster.cc:1428] Setting key 9d98e445a06a1cd773291d14e6582453
Aug 11 02:51:50 dist-test-slave-9gcw krb5kdc[12853](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416709, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.138.195@KRBTEST.COM for kudu/127.12.138.254@KRBTEST.COM
I20260811 02:51:50.012799 12957 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:51:49.999070 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.138.195:39675 (local address 127.12.138.254:37061)
0811 02:51:49.999218 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0811 02:51:49.999222 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 02:51:50.000221 (+ 999us) server_negotiation.cc:407] Connection header received
0811 02:51:50.001864 (+ 1643us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:51:50.001867 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:51:50.001943 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:51:50.002025 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:51:50.003793 (+ 1768us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:51:50.004393 (+ 600us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:51:50.005879 (+ 1486us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:51:50.006100 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:51:50.006186 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:51:50.008913 (+ 2727us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:51:50.008916 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:51:50.008918 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:51:50.008945 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:51:50.010591 (+ 1646us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:51:50.011171 (+ 580us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:51:50.011174 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:51:50.011175 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:51:50.011269 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:51:50.011634 (+ 365us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:51:50.011639 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:51:50.011641 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:51:50.011840 (+ 199us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:51:50.011951 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0811 02:51:50.012481 (+ 530us) server_negotiation.cc:299] Negotiation successful
0811 02:51:50.012614 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260811 02:51:50.013795 13453 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:51:49.999523 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.138.254:37061 (local address 127.12.138.195:39675)
0811 02:51:49.999997 (+ 474us) negotiation.cc:107] Waiting for socket to connect
0811 02:51:50.000055 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0811 02:51:50.001593 (+ 1538us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:51:50.002252 (+ 659us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:51:50.002260 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:51:50.002824 (+ 564us) client_negotiation.cc:189] Negotiated authn=SASL
0811 02:51:50.003589 (+ 765us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:51:50.003612 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:51:50.004558 (+ 946us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:51:50.004561 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:51:50.005732 (+ 1171us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:51:50.005744 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:51:50.006272 (+ 528us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:51:50.006275 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:51:50.006456 (+ 181us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:51:50.007114 (+ 658us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 02:51:50.007138 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0811 02:51:50.008709 (+ 1571us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 02:51:50.010738 (+ 2029us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:51:50.010744 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:51:50.010763 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:51:50.011041 (+ 278us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:51:50.011372 (+ 331us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:51:50.011375 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:51:50.011377 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:51:50.011534 (+ 157us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:51:50.011951 (+ 417us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 02:51:50.011959 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 02:51:50.012238 (+ 279us) client_negotiation.cc:769] Sending connection context
0811 02:51:50.012440 (+ 202us) client_negotiation.cc:240] Negotiation successful
0811 02:51:50.012815 (+ 375us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":276,"thread_start_us":132,"threads_started":1}
I20260811 02:51:50.015233 13450 heartbeater.cc:347] Connected to a master server at 127.12.138.254:37061
I20260811 02:51:50.015527 13450 heartbeater.cc:464] Registering TS with master...
I20260811 02:51:50.016189 13450 heartbeater.cc:511] Master 127.12.138.254:37061 requested a full tablet report, sending...
I20260811 02:51:50.017271 12910 ts_manager.cc:194] Registered new tserver with Master: a043f3ac80984b90a7bc97ac6bfcf9af (127.12.138.195:36069)
I20260811 02:51:50.017841 12910 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.12.138.195@KRBTEST.COM'} at 127.12.138.195:39675
I20260811 02:51:50.028087 12843 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 11 02:51:50 dist-test-slave-9gcw krb5kdc[12853](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416710, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 11 02:51:50 dist-test-slave-9gcw krb5kdc[12853](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416710, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.12.138.193@KRBTEST.COM
I20260811 02:51:50.058320 13460 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:51:50.047294 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58124 (local address 127.12.138.193:37807)
0811 02:51:50.047633 (+ 339us) server_negotiation.cc:206] Beginning negotiation
0811 02:51:50.047640 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 02:51:50.047671 (+ 31us) server_negotiation.cc:407] Connection header received
0811 02:51:50.047817 (+ 146us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:51:50.047823 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:51:50.048025 (+ 202us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:51:50.048155 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:51:50.048840 (+ 685us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:51:50.049828 (+ 988us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:51:50.050799 (+ 971us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:51:50.051262 (+ 463us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:51:50.051355 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:51:50.054239 (+ 2884us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:51:50.054249 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:51:50.054265 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:51:50.054295 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:51:50.056171 (+ 1876us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:51:50.056778 (+ 607us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:51:50.056784 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:51:50.056786 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:51:50.056887 (+ 101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:51:50.057251 (+ 364us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:51:50.057255 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:51:50.057256 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:51:50.057548 (+ 292us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:51:50.057817 (+ 269us) server_negotiation.cc:1035] Waiting for connection context
0811 02:51:50.057913 (+ 96us) server_negotiation.cc:299] Negotiation successful
0811 02:51:50.058096 (+ 183us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":237,"thread_start_us":159,"threads_started":1}
W20260811 02:51:50.059501 13036 server_base.cc:1116] Unauthorized access attempt to method kudu.tserver.TabletServerService.ListTablets from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:58124
Aug 11 02:51:50 dist-test-slave-9gcw krb5kdc[12853](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416710, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 11 02:51:50 dist-test-slave-9gcw krb5kdc[12853](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416710, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.12.138.193@KRBTEST.COM
I20260811 02:51:50.090199 13460 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:51:50.080046 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58136 (local address 127.12.138.193:37807)
0811 02:51:50.080225 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0811 02:51:50.080229 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 02:51:50.080308 (+ 79us) server_negotiation.cc:407] Connection header received
0811 02:51:50.080443 (+ 135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:51:50.080445 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:51:50.080490 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:51:50.080565 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:51:50.081408 (+ 843us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:51:50.081919 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:51:50.082841 (+ 922us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:51:50.083110 (+ 269us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:51:50.083215 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:51:50.086256 (+ 3041us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:51:50.086260 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:51:50.086262 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:51:50.086293 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:51:50.088307 (+ 2014us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:51:50.088779 (+ 472us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:51:50.088781 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:51:50.088783 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:51:50.088865 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:51:50.089127 (+ 262us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:51:50.089129 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:51:50.089130 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:51:50.089391 (+ 261us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:51:50.089512 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0811 02:51:50.089918 (+ 406us) server_negotiation.cc:299] Negotiation successful
0811 02:51:50.090034 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260811 02:51:50.091475 12843 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDawEwS/build/debug/bin/kudu with pid 12989
I20260811 02:51:50.101872 12843 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDawEwS/build/debug/bin/kudu with pid 13146
I20260811 02:51:50.108969 12843 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDawEwS/build/debug/bin/kudu with pid 13298
I20260811 02:51:50.120100 12843 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDawEwS/build/debug/bin/kudu with pid 12870
2026-08-11T02:51:50Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnListTablets (2409 ms)
[ RUN ] SecurityITest.TestAuthorizationOnChecksum
Loading random data
Initializing database '/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786416707705494-12843-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 02:51:50 dist-test-slave-9gcw krb5kdc[13467](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 02:51:50 dist-test-slave-9gcw krb5kdc[13467](info): set up 2 sockets
Aug 11 02:51:50 dist-test-slave-9gcw krb5kdc[13467](info): commencing operation
krb5kdc: starting...
W20260811 02:51:51.253113 12843 mini_kdc.cc:121] Time spent starting KDC: real 1.106s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786416707705494-12843-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786416707705494-12843-0/krb5kdc/test-user.keytab.
Aug 11 02:51:51 dist-test-slave-9gcw krb5kdc[13467](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416711, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T02:51:51Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T02:51:51Z Disabled control of system clock
WARNING: no policy specified for kudu/127.12.138.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.138.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.138.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786416707705494-12843-0/krb5kdc/kudu_127.12.138.254.keytab.
Entry for principal kudu/127.12.138.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786416707705494-12843-0/krb5kdc/kudu_127.12.138.254.keytab.
I20260811 02:51:51.388631 12843 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDawEwS/build/debug/bin/kudu
/tmp/dist-test-taskDawEwS/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786416707705494-12843-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786416707705494-12843-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786416707705494-12843-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786416707705494-12843-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.138.254:34899
--webserver_interface=127.12.138.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786416707705494-12843-0/krb5kdc/kudu_127.12.138.254.keytab
--principal=kudu/127.12.138.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.138.212:43363
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.12.138.254:34899
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786416707705494-12843-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786416707705494-12843-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786416707705494-12843-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 02:51:51.507781 13481 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 02:51:51.508157 13481 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 02:51:51.508255 13481 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 02:51:51.512436 13481 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 02:51:51.512558 13481 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 02:51:51.512614 13481 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 02:51:51.512681 13481 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 02:51:51.512724 13481 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 02:51:51.518143 13481 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.138.212:43363
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786416707705494-12843-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786416707705494-12843-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.12.138.254:34899
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786416707705494-12843-0/krb5kdc/kudu_127.12.138.254.keytab
--principal=kudu/127.12.138.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.138.254:34899
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786416707705494-12843-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.12.138.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13481
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786416707705494-12843-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 02:09:49 UTC on ccycloud.quasar-fsnqqc.root.comops.site
build id 4579570
I20260811 02:51:51.519282 13481 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 02:51:51.520358 13481 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 02:51:51.529486 13487 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:51:51.529831 13489 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:51:51.529960 13481 server_base.cc:1034] running on GCE node
W20260811 02:51:51.529979 13486 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:51:51.530611 13481 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 02:51:51.531796 13481 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 02:51:51.532989 13481 hybrid_clock.cc:648] HybridClock initialized: now 1786416711532968 us; error 38 us; skew 500 ppm
Aug 11 02:51:51 dist-test-slave-9gcw krb5kdc[13467](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416711, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.138.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 02:51:51.536063 13481 init.cc:377] Logged in from keytab as kudu/127.12.138.254@KRBTEST.COM (short username kudu)
I20260811 02:51:51.537082 13481 webserver.cc:460] Webserver started at http://127.12.138.254:37917/ using document root <none> and password file <none>
I20260811 02:51:51.537736 13481 fs_manager.cc:359] Metadata directory not provided
I20260811 02:51:51.537802 13481 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 02:51:51.538031 13481 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 02:51:51.540223 13481 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786416707705494-12843-0/minicluster-data/master-0/data/instance:
uuid: "c1afecbc02fa4aedaf1c8bf2d9972232"
format_stamp: "Formatted at 2026-08-11 02:51:51 on dist-test-slave-9gcw"
server_key: "041d452a25ac8b7c5eab4bc246717d7e"
server_key_iv: "7143858aefe4f4dbb602e83ab0d6ade1"
server_key_version: "encryptionkey@0"
I20260811 02:51:51.540822 13481 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786416707705494-12843-0/minicluster-data/master-0/wal/instance:
uuid: "c1afecbc02fa4aedaf1c8bf2d9972232"
format_stamp: "Formatted at 2026-08-11 02:51:51 on dist-test-slave-9gcw"
server_key: "041d452a25ac8b7c5eab4bc246717d7e"
server_key_iv: "7143858aefe4f4dbb602e83ab0d6ade1"
server_key_version: "encryptionkey@0"
I20260811 02:51:51.544983 13481 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.007s sys 0.000s
I20260811 02:51:51.547884 13497 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 02:51:51.549499 13481 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 02:51:51.549660 13481 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786416707705494-12843-0/minicluster-data/master-0/data,/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786416707705494-12843-0/minicluster-data/master-0/wal
uuid: "c1afecbc02fa4aedaf1c8bf2d9972232"
format_stamp: "Formatted at 2026-08-11 02:51:51 on dist-test-slave-9gcw"
server_key: "041d452a25ac8b7c5eab4bc246717d7e"
server_key_iv: "7143858aefe4f4dbb602e83ab0d6ade1"
server_key_version: "encryptionkey@0"
I20260811 02:51:51.549767 13481 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786416707705494-12843-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786416707705494-12843-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786416707705494-12843-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 02:51:51.561684 13481 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 02:51:51.565258 13481 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 02:51:51.565515 13481 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 02:51:51.574424 13481 rpc_server.cc:307] RPC server started. Bound to: 127.12.138.254:34899
I20260811 02:51:51.574458 13558 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.138.254:34899 every 8 connection(s)
I20260811 02:51:51.575574 13481 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786416707705494-12843-0/minicluster-data/master-0/data/info.pb
I20260811 02:51:51.576707 12843 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDawEwS/build/debug/bin/kudu as pid 13481
I20260811 02:51:51.576844 12843 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786416707705494-12843-0/minicluster-data/master-0/wal/instance
I20260811 02:51:51.577221 12843 external_mini_cluster.cc:1428] Setting key 2e376f000f86a156748161e86c5b5754
I20260811 02:51:51.579406 13559 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 02:51:51.585613 13559 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c1afecbc02fa4aedaf1c8bf2d9972232: Bootstrap starting.
Aug 11 02:51:51 dist-test-slave-9gcw krb5kdc[13467](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416711, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.12.138.254@KRBTEST.COM
I20260811 02:51:51.588138 13559 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c1afecbc02fa4aedaf1c8bf2d9972232: Neither blocks nor log segments found. Creating new log.
I20260811 02:51:51.588949 13559 log.cc:824] T 00000000000000000000000000000000 P c1afecbc02fa4aedaf1c8bf2d9972232: Log is configured to *not* fsync() on all Append() calls
I20260811 02:51:51.591153 13559 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c1afecbc02fa4aedaf1c8bf2d9972232: No bootstrap required, opened a new log
I20260811 02:51:51.593255 13562 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:51:51.578851 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53876 (local address 127.12.138.254:34899)
0811 02:51:51.579354 (+ 503us) server_negotiation.cc:206] Beginning negotiation
0811 02:51:51.579373 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0811 02:51:51.579414 (+ 41us) server_negotiation.cc:407] Connection header received
0811 02:51:51.580268 (+ 854us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:51:51.580288 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:51:51.580659 (+ 371us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:51:51.581056 (+ 397us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:51:51.582099 (+ 1043us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:51:51.583254 (+ 1155us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:51:51.584263 (+ 1009us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:51:51.584820 (+ 557us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:51:51.585042 (+ 222us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:51:51.587945 (+ 2903us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:51:51.587950 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:51:51.587963 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:51:51.587992 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:51:51.590064 (+ 2072us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:51:51.590536 (+ 472us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:51:51.590541 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:51:51.590545 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:51:51.590655 (+ 110us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:51:51.590952 (+ 297us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:51:51.590957 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:51:51.590958 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:51:51.591448 (+ 490us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:51:51.591651 (+ 203us) server_negotiation.cc:1035] Waiting for connection context
0811 02:51:51.591965 (+ 314us) server_negotiation.cc:299] Negotiation successful
0811 02:51:51.592264 (+ 299us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":345,"thread_start_us":153,"threads_started":1}
I20260811 02:51:51.595010 13559 raft_consensus.cc:348] T 00000000000000000000000000000000 P c1afecbc02fa4aedaf1c8bf2d9972232 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c1afecbc02fa4aedaf1c8bf2d9972232" member_type: VOTER last_known_addr { host: "127.12.138.254" port: 34899 } }
I20260811 02:51:51.595319 13559 raft_consensus.cc:374] T 00000000000000000000000000000000 P c1afecbc02fa4aedaf1c8bf2d9972232 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 02:51:51.595422 13559 raft_consensus.cc:729] T 00000000000000000000000000000000 P c1afecbc02fa4aedaf1c8bf2d9972232 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c1afecbc02fa4aedaf1c8bf2d9972232, State: Initialized, Role: FOLLOWER
I20260811 02:51:51.595957 13559 consensus_queue.cc:260] T 00000000000000000000000000000000 P c1afecbc02fa4aedaf1c8bf2d9972232 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c1afecbc02fa4aedaf1c8bf2d9972232" member_type: VOTER last_known_addr { host: "127.12.138.254" port: 34899 } }
I20260811 02:51:51.596146 13559 raft_consensus.cc:388] T 00000000000000000000000000000000 P c1afecbc02fa4aedaf1c8bf2d9972232 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 02:51:51.596242 13559 raft_consensus.cc:482] T 00000000000000000000000000000000 P c1afecbc02fa4aedaf1c8bf2d9972232 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 02:51:51.596369 13559 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c1afecbc02fa4aedaf1c8bf2d9972232 [term 0 FOLLOWER]: Advancing to term 1
I20260811 02:51:51.597668 13559 raft_consensus.cc:504] T 00000000000000000000000000000000 P c1afecbc02fa4aedaf1c8bf2d9972232 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c1afecbc02fa4aedaf1c8bf2d9972232" member_type: VOTER last_known_addr { host: "127.12.138.254" port: 34899 } }
I20260811 02:51:51.598034 13559 leader_election.cc:302] T 00000000000000000000000000000000 P c1afecbc02fa4aedaf1c8bf2d9972232 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c1afecbc02fa4aedaf1c8bf2d9972232; no voters:
I20260811 02:51:51.598380 13559 leader_election.cc:288] T 00000000000000000000000000000000 P c1afecbc02fa4aedaf1c8bf2d9972232 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 02:51:51.598513 13564 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c1afecbc02fa4aedaf1c8bf2d9972232 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 02:51:51.598789 13564 raft_consensus.cc:686] T 00000000000000000000000000000000 P c1afecbc02fa4aedaf1c8bf2d9972232 [term 1 LEADER]: Becoming Leader. State: Replica: c1afecbc02fa4aedaf1c8bf2d9972232, State: Running, Role: LEADER
I20260811 02:51:51.599089 13564 consensus_queue.cc:237] T 00000000000000000000000000000000 P c1afecbc02fa4aedaf1c8bf2d9972232 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c1afecbc02fa4aedaf1c8bf2d9972232" member_type: VOTER last_known_addr { host: "127.12.138.254" port: 34899 } }
I20260811 02:51:51.599769 13559 sys_catalog.cc:564] T 00000000000000000000000000000000 P c1afecbc02fa4aedaf1c8bf2d9972232 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 02:51:51.600840 13565 sys_catalog.cc:455] T 00000000000000000000000000000000 P c1afecbc02fa4aedaf1c8bf2d9972232 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c1afecbc02fa4aedaf1c8bf2d9972232" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c1afecbc02fa4aedaf1c8bf2d9972232" member_type: VOTER last_known_addr { host: "127.12.138.254" port: 34899 } } }
I20260811 02:51:51.600986 13565 sys_catalog.cc:458] T 00000000000000000000000000000000 P c1afecbc02fa4aedaf1c8bf2d9972232 [sys.catalog]: This master's current role is: LEADER
I20260811 02:51:51.601506 13566 sys_catalog.cc:455] T 00000000000000000000000000000000 P c1afecbc02fa4aedaf1c8bf2d9972232 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c1afecbc02fa4aedaf1c8bf2d9972232. Latest consensus state: current_term: 1 leader_uuid: "c1afecbc02fa4aedaf1c8bf2d9972232" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c1afecbc02fa4aedaf1c8bf2d9972232" member_type: VOTER last_known_addr { host: "127.12.138.254" port: 34899 } } }
I20260811 02:51:51.601606 13566 sys_catalog.cc:458] T 00000000000000000000000000000000 P c1afecbc02fa4aedaf1c8bf2d9972232 [sys.catalog]: This master's current role is: LEADER
I20260811 02:51:51.601912 13576 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 02:51:51.605283 13576 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 02:51:51.611663 13576 catalog_manager.cc:1347] Generated new cluster ID: 529b94dbf13b488dba84d063d476f869
I20260811 02:51:51.611748 13576 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 02:51:51.640021 13576 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 02:51:51.641253 13576 catalog_manager.cc:1504] Loading token signing keys...
I20260811 02:51:51.649895 13576 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c1afecbc02fa4aedaf1c8bf2d9972232: Generated new TSK 0
I20260811 02:51:51.650642 13576 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.12.138.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.138.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.138.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786416707705494-12843-0/krb5kdc/kudu_127.12.138.193.keytab.
Entry for principal kudu/127.12.138.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786416707705494-12843-0/krb5kdc/kudu_127.12.138.193.keytab.
I20260811 02:51:51.689224 12843 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDawEwS/build/debug/bin/kudu
/tmp/dist-test-taskDawEwS/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786416707705494-12843-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786416707705494-12843-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786416707705494-12843-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786416707705494-12843-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.138.193:0
--local_ip_for_outbound_sockets=127.12.138.193
--webserver_interface=127.12.138.193
--webserver_port=0
--tserver_master_addrs=127.12.138.254:34899
--keytab_file=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786416707705494-12843-0/krb5kdc/kudu_127.12.138.193.keytab
--principal=kudu/127.12.138.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.138.212:43363
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786416707705494-12843-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786416707705494-12843-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786416707705494-12843-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 02:51:51.831136 13589 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 02:51:51.831472 13589 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 02:51:51.831552 13589 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 02:51:51.835676 13589 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 02:51:51.835774 13589 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 02:51:51.835865 13589 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.138.193
I20260811 02:51:51.841281 13589 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.138.212:43363
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786416707705494-12843-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786416707705494-12843-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786416707705494-12843-0/krb5kdc/kudu_127.12.138.193.keytab
--principal=kudu/127.12.138.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.138.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786416707705494-12843-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.12.138.193
--webserver_port=0
--tserver_master_addrs=127.12.138.254:34899
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13589
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.138.193
--log_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786416707705494-12843-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 02:09:49 UTC on ccycloud.quasar-fsnqqc.root.comops.site
build id 4579570
I20260811 02:51:51.842536 13589 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 02:51:51.843593 13589 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 02:51:51.853794 13594 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:51:51.853889 13597 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:51:51.853793 13595 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:51:51.854228 13589 server_base.cc:1034] running on GCE node
I20260811 02:51:51.854813 13589 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 02:51:51.855552 13589 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 02:51:51.856714 13589 hybrid_clock.cc:648] HybridClock initialized: now 1786416711856699 us; error 28 us; skew 500 ppm
Aug 11 02:51:51 dist-test-slave-9gcw krb5kdc[13467](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416711, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.138.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 02:51:51.859679 13589 init.cc:377] Logged in from keytab as kudu/127.12.138.193@KRBTEST.COM (short username kudu)
I20260811 02:51:51.860598 13589 webserver.cc:460] Webserver started at http://127.12.138.193:41325/ using document root <none> and password file <none>
I20260811 02:51:51.861234 13589 fs_manager.cc:359] Metadata directory not provided
I20260811 02:51:51.861302 13589 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 02:51:51.861486 13589 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 02:51:51.863587 13589 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786416707705494-12843-0/minicluster-data/ts-0/data/instance:
uuid: "6a07dfd0984340b2a25c3fed608b86a4"
format_stamp: "Formatted at 2026-08-11 02:51:51 on dist-test-slave-9gcw"
server_key: "59a07e8e5071dca5248304f3183bcb06"
server_key_iv: "992773525fa2f3b7ad222d166cc46e3c"
server_key_version: "encryptionkey@0"
I20260811 02:51:51.864137 13589 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786416707705494-12843-0/minicluster-data/ts-0/wal/instance:
uuid: "6a07dfd0984340b2a25c3fed608b86a4"
format_stamp: "Formatted at 2026-08-11 02:51:51 on dist-test-slave-9gcw"
server_key: "59a07e8e5071dca5248304f3183bcb06"
server_key_iv: "992773525fa2f3b7ad222d166cc46e3c"
server_key_version: "encryptionkey@0"
I20260811 02:51:51.868121 13589 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.004s
I20260811 02:51:51.870673 13606 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 02:51:51.872109 13589 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260811 02:51:51.872269 13589 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786416707705494-12843-0/minicluster-data/ts-0/data,/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786416707705494-12843-0/minicluster-data/ts-0/wal
uuid: "6a07dfd0984340b2a25c3fed608b86a4"
format_stamp: "Formatted at 2026-08-11 02:51:51 on dist-test-slave-9gcw"
server_key: "59a07e8e5071dca5248304f3183bcb06"
server_key_iv: "992773525fa2f3b7ad222d166cc46e3c"
server_key_version: "encryptionkey@0"
I20260811 02:51:51.872408 13589 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786416707705494-12843-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786416707705494-12843-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786416707705494-12843-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 02:51:51.887326 13589 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 02:51:51.890576 13589 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 02:51:51.890805 13589 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 02:51:51.891444 13589 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 02:51:51.892331 13589 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 02:51:51.892418 13589 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 02:51:51.892498 13589 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 02:51:51.892549 13589 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 02:51:51.902463 13589 rpc_server.cc:307] RPC server started. Bound to: 127.12.138.193:35251
I20260811 02:51:51.902490 13741 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.138.193:35251 every 8 connection(s)
I20260811 02:51:51.903424 13589 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786416707705494-12843-0/minicluster-data/ts-0/data/info.pb
I20260811 02:51:51.908125 12843 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDawEwS/build/debug/bin/kudu as pid 13589
I20260811 02:51:51.908250 12843 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786416707705494-12843-0/minicluster-data/ts-0/wal/instance
I20260811 02:51:51.908541 12843 external_mini_cluster.cc:1428] Setting key 738a54a47a5bf68f0ea92ed93211e12c
Aug 11 02:51:51 dist-test-slave-9gcw krb5kdc[13467](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416711, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.138.193@KRBTEST.COM for kudu/127.12.138.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.12.138.194@KRBTEST.COM; defaulting to no policy
I20260811 02:51:51.919312 13562 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:51:51.905516 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.138.193:54323 (local address 127.12.138.254:34899)
0811 02:51:51.905676 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0811 02:51:51.905680 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 02:51:51.906523 (+ 843us) server_negotiation.cc:407] Connection header received
0811 02:51:51.907792 (+ 1269us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:51:51.907795 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:51:51.907859 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:51:51.907938 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:51:51.909634 (+ 1696us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:51:51.910482 (+ 848us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:51:51.912258 (+ 1776us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:51:51.912510 (+ 252us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:51:51.912623 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:51:51.915558 (+ 2935us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:51:51.915562 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:51:51.915565 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:51:51.915594 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:51:51.917123 (+ 1529us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:51:51.917705 (+ 582us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:51:51.917710 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:51:51.917711 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:51:51.917797 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:51:51.918142 (+ 345us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:51:51.918146 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:51:51.918148 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:51:51.918353 (+ 205us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:51:51.918466 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0811 02:51:51.918987 (+ 521us) server_negotiation.cc:299] Negotiation successful
0811 02:51:51.919125 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260811 02:51:51.920132 13747 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:51:51.905903 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.138.254:34899 (local address 127.12.138.193:54323)
0811 02:51:51.906364 (+ 461us) negotiation.cc:107] Waiting for socket to connect
0811 02:51:51.906407 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0811 02:51:51.907586 (+ 1179us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:51:51.908183 (+ 597us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:51:51.908195 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:51:51.908699 (+ 504us) client_negotiation.cc:189] Negotiated authn=SASL
0811 02:51:51.909427 (+ 728us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:51:51.909468 (+ 41us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:51:51.910656 (+ 1188us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:51:51.910660 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:51:51.912113 (+ 1453us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:51:51.912125 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:51:51.912670 (+ 545us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:51:51.912674 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:51:51.912858 (+ 184us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:51:51.913572 (+ 714us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 02:51:51.913598 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0811 02:51:51.915367 (+ 1769us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 02:51:51.917305 (+ 1938us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:51:51.917311 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:51:51.917328 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:51:51.917596 (+ 268us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:51:51.917892 (+ 296us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:51:51.917895 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:51:51.917897 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:51:51.918045 (+ 148us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:51:51.918466 (+ 421us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 02:51:51.918474 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 02:51:51.918748 (+ 274us) client_negotiation.cc:769] Sending connection context
0811 02:51:51.918947 (+ 199us) client_negotiation.cc:240] Negotiation successful
0811 02:51:51.919284 (+ 337us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":273,"thread_start_us":108,"threads_started":1}
I20260811 02:51:51.921460 13742 heartbeater.cc:347] Connected to a master server at 127.12.138.254:34899
I20260811 02:51:51.921780 13742 heartbeater.cc:464] Registering TS with master...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.138.194@KRBTEST.COM" created.
I20260811 02:51:51.922418 13742 heartbeater.cc:511] Master 127.12.138.254:34899 requested a full tablet report, sending...
I20260811 02:51:51.924281 13520 ts_manager.cc:194] Registered new tserver with Master: 6a07dfd0984340b2a25c3fed608b86a4 (127.12.138.193:35251)
I20260811 02:51:51.926101 13520 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.12.138.193@KRBTEST.COM'} at 127.12.138.193:54323
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.138.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786416707705494-12843-0/krb5kdc/kudu_127.12.138.194.keytab.
Entry for principal kudu/127.12.138.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786416707705494-12843-0/krb5kdc/kudu_127.12.138.194.keytab.
I20260811 02:51:51.942554 12843 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDawEwS/build/debug/bin/kudu
/tmp/dist-test-taskDawEwS/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786416707705494-12843-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786416707705494-12843-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786416707705494-12843-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786416707705494-12843-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.138.194:0
--local_ip_for_outbound_sockets=127.12.138.194
--webserver_interface=127.12.138.194
--webserver_port=0
--tserver_master_addrs=127.12.138.254:34899
--keytab_file=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786416707705494-12843-0/krb5kdc/kudu_127.12.138.194.keytab
--principal=kudu/127.12.138.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.138.212:43363
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786416707705494-12843-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786416707705494-12843-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786416707705494-12843-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 02:51:52.063054 13750 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 02:51:52.063341 13750 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 02:51:52.063388 13750 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 02:51:52.067523 13750 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 02:51:52.067593 13750 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 02:51:52.067657 13750 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.138.194
I20260811 02:51:52.073045 13750 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.138.212:43363
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786416707705494-12843-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786416707705494-12843-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786416707705494-12843-0/krb5kdc/kudu_127.12.138.194.keytab
--principal=kudu/127.12.138.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.138.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786416707705494-12843-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.12.138.194
--webserver_port=0
--tserver_master_addrs=127.12.138.254:34899
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13750
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.138.194
--log_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786416707705494-12843-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 02:09:49 UTC on ccycloud.quasar-fsnqqc.root.comops.site
build id 4579570
I20260811 02:51:52.074236 13750 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 02:51:52.075359 13750 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 02:51:52.085992 13759 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:51:52.086102 13762 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:51:52.085994 13758 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:51:52.086283 13750 server_base.cc:1034] running on GCE node
I20260811 02:51:52.086812 13750 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 02:51:52.087575 13750 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 02:51:52.088783 13750 hybrid_clock.cc:648] HybridClock initialized: now 1786416712088753 us; error 53 us; skew 500 ppm
Aug 11 02:51:52 dist-test-slave-9gcw krb5kdc[13467](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416712, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.138.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 02:51:52.091924 13750 init.cc:377] Logged in from keytab as kudu/127.12.138.194@KRBTEST.COM (short username kudu)
I20260811 02:51:52.093027 13750 webserver.cc:460] Webserver started at http://127.12.138.194:39617/ using document root <none> and password file <none>
I20260811 02:51:52.093699 13750 fs_manager.cc:359] Metadata directory not provided
I20260811 02:51:52.093796 13750 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 02:51:52.094039 13750 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 02:51:52.096230 13750 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786416707705494-12843-0/minicluster-data/ts-1/data/instance:
uuid: "b1757237a2a243e492e295d2389f485d"
format_stamp: "Formatted at 2026-08-11 02:51:52 on dist-test-slave-9gcw"
server_key: "2de65b82e0700c5488203bda2cf8b25a"
server_key_iv: "91c531a3a182fb01182e6d7e7f900d63"
server_key_version: "encryptionkey@0"
I20260811 02:51:52.096834 13750 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786416707705494-12843-0/minicluster-data/ts-1/wal/instance:
uuid: "b1757237a2a243e492e295d2389f485d"
format_stamp: "Formatted at 2026-08-11 02:51:52 on dist-test-slave-9gcw"
server_key: "2de65b82e0700c5488203bda2cf8b25a"
server_key_iv: "91c531a3a182fb01182e6d7e7f900d63"
server_key_version: "encryptionkey@0"
I20260811 02:51:52.100997 13750 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.001s
I20260811 02:51:52.103941 13769 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 02:51:52.105432 13750 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260811 02:51:52.105583 13750 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786416707705494-12843-0/minicluster-data/ts-1/data,/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786416707705494-12843-0/minicluster-data/ts-1/wal
uuid: "b1757237a2a243e492e295d2389f485d"
format_stamp: "Formatted at 2026-08-11 02:51:52 on dist-test-slave-9gcw"
server_key: "2de65b82e0700c5488203bda2cf8b25a"
server_key_iv: "91c531a3a182fb01182e6d7e7f900d63"
server_key_version: "encryptionkey@0"
I20260811 02:51:52.105715 13750 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786416707705494-12843-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786416707705494-12843-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786416707705494-12843-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 02:51:52.116570 13750 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 02:51:52.119940 13750 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 02:51:52.120214 13750 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 02:51:52.120920 13750 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 02:51:52.121881 13750 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 02:51:52.121968 13750 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 02:51:52.122043 13750 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 02:51:52.122097 13750 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 02:51:52.131944 13750 rpc_server.cc:307] RPC server started. Bound to: 127.12.138.194:41449
I20260811 02:51:52.132017 13901 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.138.194:41449 every 8 connection(s)
I20260811 02:51:52.132998 13750 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786416707705494-12843-0/minicluster-data/ts-1/data/info.pb
I20260811 02:51:52.140693 12843 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDawEwS/build/debug/bin/kudu as pid 13750
I20260811 02:51:52.140805 12843 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786416707705494-12843-0/minicluster-data/ts-1/wal/instance
I20260811 02:51:52.141125 12843 external_mini_cluster.cc:1428] Setting key 07cc71a8ca5a267ea20a11f006d29870
Aug 11 02:51:52 dist-test-slave-9gcw krb5kdc[13467](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416712, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.138.194@KRBTEST.COM for kudu/127.12.138.254@KRBTEST.COM
I20260811 02:51:52.148425 13562 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:51:52.135038 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.138.194:50035 (local address 127.12.138.254:34899)
0811 02:51:52.135182 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0811 02:51:52.135186 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 02:51:52.136113 (+ 927us) server_negotiation.cc:407] Connection header received
0811 02:51:52.137475 (+ 1362us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:51:52.137478 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:51:52.137527 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:51:52.137613 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:51:52.138851 (+ 1238us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:51:52.139404 (+ 553us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:51:52.140620 (+ 1216us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:51:52.140966 (+ 346us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:51:52.141056 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:51:52.144042 (+ 2986us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:51:52.144046 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:51:52.144048 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:51:52.144082 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:51:52.146088 (+ 2006us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:51:52.146595 (+ 507us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:51:52.146601 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:51:52.146606 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:51:52.146710 (+ 104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:51:52.147107 (+ 397us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:51:52.147113 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:51:52.147117 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:51:52.147394 (+ 277us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:51:52.147506 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0811 02:51:52.148011 (+ 505us) server_negotiation.cc:299] Negotiation successful
0811 02:51:52.148172 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260811 02:51:52.149252 13906 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:51:52.135494 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.138.254:34899 (local address 127.12.138.194:50035)
0811 02:51:52.135947 (+ 453us) negotiation.cc:107] Waiting for socket to connect
0811 02:51:52.135989 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0811 02:51:52.137129 (+ 1140us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:51:52.137824 (+ 695us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:51:52.137831 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:51:52.138196 (+ 365us) client_negotiation.cc:189] Negotiated authn=SASL
0811 02:51:52.138694 (+ 498us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:51:52.138707 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:51:52.139521 (+ 814us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:51:52.139524 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:51:52.140482 (+ 958us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:51:52.140496 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:51:52.141128 (+ 632us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:51:52.141133 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:51:52.141419 (+ 286us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:51:52.142190 (+ 771us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 02:51:52.142209 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0811 02:51:52.143873 (+ 1664us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 02:51:52.146204 (+ 2331us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:51:52.146209 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:51:52.146221 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:51:52.146493 (+ 272us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:51:52.146852 (+ 359us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:51:52.146856 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:51:52.146858 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:51:52.147005 (+ 147us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:51:52.147505 (+ 500us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 02:51:52.147510 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 02:51:52.147752 (+ 242us) client_negotiation.cc:769] Sending connection context
0811 02:51:52.147914 (+ 162us) client_negotiation.cc:240] Negotiation successful
0811 02:51:52.148238 (+ 324us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":268,"thread_start_us":123,"threads_started":1}
I20260811 02:51:52.150559 13903 heartbeater.cc:347] Connected to a master server at 127.12.138.254:34899
I20260811 02:51:52.150842 13903 heartbeater.cc:464] Registering TS with master...
I20260811 02:51:52.151472 13903 heartbeater.cc:511] Master 127.12.138.254:34899 requested a full tablet report, sending...
I20260811 02:51:52.152618 13520 ts_manager.cc:194] Registered new tserver with Master: b1757237a2a243e492e295d2389f485d (127.12.138.194:41449)
WARNING: no policy specified for kudu/127.12.138.195@KRBTEST.COM; defaulting to no policy
I20260811 02:51:52.153322 13520 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.12.138.194@KRBTEST.COM'} at 127.12.138.194:50035
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.138.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.138.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786416707705494-12843-0/krb5kdc/kudu_127.12.138.195.keytab.
Entry for principal kudu/127.12.138.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786416707705494-12843-0/krb5kdc/kudu_127.12.138.195.keytab.
I20260811 02:51:52.173063 12843 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDawEwS/build/debug/bin/kudu
/tmp/dist-test-taskDawEwS/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786416707705494-12843-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786416707705494-12843-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786416707705494-12843-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786416707705494-12843-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.138.195:0
--local_ip_for_outbound_sockets=127.12.138.195
--webserver_interface=127.12.138.195
--webserver_port=0
--tserver_master_addrs=127.12.138.254:34899
--keytab_file=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786416707705494-12843-0/krb5kdc/kudu_127.12.138.195.keytab
--principal=kudu/127.12.138.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.138.212:43363
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786416707705494-12843-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786416707705494-12843-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786416707705494-12843-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 02:51:52.293741 13909 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 02:51:52.294039 13909 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 02:51:52.294112 13909 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 02:51:52.298408 13909 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 02:51:52.298476 13909 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 02:51:52.298566 13909 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.138.195
I20260811 02:51:52.304251 13909 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.138.212:43363
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786416707705494-12843-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786416707705494-12843-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786416707705494-12843-0/krb5kdc/kudu_127.12.138.195.keytab
--principal=kudu/127.12.138.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.138.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786416707705494-12843-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.12.138.195
--webserver_port=0
--tserver_master_addrs=127.12.138.254:34899
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13909
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.138.195
--log_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786416707705494-12843-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 02:09:49 UTC on ccycloud.quasar-fsnqqc.root.comops.site
build id 4579570
I20260811 02:51:52.305516 13909 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 02:51:52.306576 13909 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 02:51:52.317140 13918 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:51:52.317119 13920 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:51:52.317251 13917 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:51:52.317730 13909 server_base.cc:1034] running on GCE node
I20260811 02:51:52.318120 13909 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 02:51:52.318872 13909 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 02:51:52.320050 13909 hybrid_clock.cc:648] HybridClock initialized: now 1786416712320012 us; error 49 us; skew 500 ppm
Aug 11 02:51:52 dist-test-slave-9gcw krb5kdc[13467](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416712, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.138.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 02:51:52.323248 13909 init.cc:377] Logged in from keytab as kudu/127.12.138.195@KRBTEST.COM (short username kudu)
I20260811 02:51:52.324307 13909 webserver.cc:460] Webserver started at http://127.12.138.195:39101/ using document root <none> and password file <none>
I20260811 02:51:52.324895 13909 fs_manager.cc:359] Metadata directory not provided
I20260811 02:51:52.324987 13909 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 02:51:52.325300 13909 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 02:51:52.327432 13909 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786416707705494-12843-0/minicluster-data/ts-2/data/instance:
uuid: "afa0d92aa4bb434a8d219af54f8eff8d"
format_stamp: "Formatted at 2026-08-11 02:51:52 on dist-test-slave-9gcw"
server_key: "f2d972e4a3b879cb45fa87930cf9b487"
server_key_iv: "1df2e10f85d725a7621f866fc669ce2e"
server_key_version: "encryptionkey@0"
I20260811 02:51:52.328019 13909 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786416707705494-12843-0/minicluster-data/ts-2/wal/instance:
uuid: "afa0d92aa4bb434a8d219af54f8eff8d"
format_stamp: "Formatted at 2026-08-11 02:51:52 on dist-test-slave-9gcw"
server_key: "f2d972e4a3b879cb45fa87930cf9b487"
server_key_iv: "1df2e10f85d725a7621f866fc669ce2e"
server_key_version: "encryptionkey@0"
I20260811 02:51:52.331916 13909 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.000s sys 0.006s
I20260811 02:51:52.334481 13930 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 02:51:52.335754 13909 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260811 02:51:52.335898 13909 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786416707705494-12843-0/minicluster-data/ts-2/data,/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786416707705494-12843-0/minicluster-data/ts-2/wal
uuid: "afa0d92aa4bb434a8d219af54f8eff8d"
format_stamp: "Formatted at 2026-08-11 02:51:52 on dist-test-slave-9gcw"
server_key: "f2d972e4a3b879cb45fa87930cf9b487"
server_key_iv: "1df2e10f85d725a7621f866fc669ce2e"
server_key_version: "encryptionkey@0"
I20260811 02:51:52.336010 13909 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786416707705494-12843-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786416707705494-12843-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786416707705494-12843-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 02:51:52.357741 13909 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 02:51:52.361225 13909 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 02:51:52.361438 13909 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 02:51:52.362054 13909 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 02:51:52.362931 13909 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 02:51:52.363010 13909 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 02:51:52.363109 13909 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 02:51:52.363160 13909 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 02:51:52.372454 13909 rpc_server.cc:307] RPC server started. Bound to: 127.12.138.195:42587
I20260811 02:51:52.372480 14058 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.138.195:42587 every 8 connection(s)
I20260811 02:51:52.373464 13909 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786416707705494-12843-0/minicluster-data/ts-2/data/info.pb
I20260811 02:51:52.381347 12843 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDawEwS/build/debug/bin/kudu as pid 13909
I20260811 02:51:52.381489 12843 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786416707705494-12843-0/minicluster-data/ts-2/wal/instance
I20260811 02:51:52.381809 12843 external_mini_cluster.cc:1428] Setting key d8f358ce899253e16fd0adb926d39ead
Aug 11 02:51:52 dist-test-slave-9gcw krb5kdc[13467](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416712, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.138.195@KRBTEST.COM for kudu/127.12.138.254@KRBTEST.COM
I20260811 02:51:52.388515 13562 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:51:52.375524 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.138.195:37249 (local address 127.12.138.254:34899)
0811 02:51:52.375760 (+ 236us) server_negotiation.cc:206] Beginning negotiation
0811 02:51:52.375767 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 02:51:52.376661 (+ 894us) server_negotiation.cc:407] Connection header received
0811 02:51:52.378202 (+ 1541us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:51:52.378205 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:51:52.378254 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:51:52.378340 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:51:52.379602 (+ 1262us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:51:52.380214 (+ 612us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:51:52.381679 (+ 1465us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:51:52.382016 (+ 337us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:51:52.382110 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:51:52.385255 (+ 3145us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:51:52.385259 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:51:52.385261 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:51:52.385288 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:51:52.386548 (+ 1260us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:51:52.387059 (+ 511us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:51:52.387062 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:51:52.387063 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:51:52.387148 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:51:52.387480 (+ 332us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:51:52.387482 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:51:52.387483 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:51:52.387671 (+ 188us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:51:52.387816 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0811 02:51:52.388287 (+ 471us) server_negotiation.cc:299] Negotiation successful
0811 02:51:52.388394 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":119}
I20260811 02:51:52.389475 14062 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:51:52.376018 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.138.254:34899 (local address 127.12.138.195:37249)
0811 02:51:52.376501 (+ 483us) negotiation.cc:107] Waiting for socket to connect
0811 02:51:52.376544 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0811 02:51:52.377936 (+ 1392us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:51:52.378504 (+ 568us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:51:52.378511 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:51:52.378909 (+ 398us) client_negotiation.cc:189] Negotiated authn=SASL
0811 02:51:52.379432 (+ 523us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:51:52.379446 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:51:52.380367 (+ 921us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:51:52.380370 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:51:52.381522 (+ 1152us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:51:52.381536 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:51:52.382186 (+ 650us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:51:52.382190 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:51:52.382384 (+ 194us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:51:52.383080 (+ 696us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 02:51:52.383097 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 02:51:52.385022 (+ 1925us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 02:51:52.386652 (+ 1630us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:51:52.386657 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:51:52.386668 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:51:52.386957 (+ 289us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:51:52.387237 (+ 280us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:51:52.387239 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:51:52.387240 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:51:52.387389 (+ 149us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:51:52.387821 (+ 432us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 02:51:52.387826 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 02:51:52.388068 (+ 242us) client_negotiation.cc:769] Sending connection context
0811 02:51:52.388231 (+ 163us) client_negotiation.cc:240] Negotiation successful
0811 02:51:52.388510 (+ 279us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":297,"thread_start_us":111,"threads_started":1}
I20260811 02:51:52.390848 14059 heartbeater.cc:347] Connected to a master server at 127.12.138.254:34899
I20260811 02:51:52.391108 14059 heartbeater.cc:464] Registering TS with master...
I20260811 02:51:52.391618 14059 heartbeater.cc:511] Master 127.12.138.254:34899 requested a full tablet report, sending...
I20260811 02:51:52.392779 13520 ts_manager.cc:194] Registered new tserver with Master: afa0d92aa4bb434a8d219af54f8eff8d (127.12.138.195:42587)
I20260811 02:51:52.393533 13520 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.12.138.195@KRBTEST.COM'} at 127.12.138.195:37249
I20260811 02:51:52.396409 12843 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 02:51:52.407613 13562 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:51:52.400163 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39690 (local address 127.12.138.254:34899)
0811 02:51:52.400321 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0811 02:51:52.400325 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 02:51:52.400485 (+ 160us) server_negotiation.cc:407] Connection header received
0811 02:51:52.400606 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:51:52.400608 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:51:52.400647 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:51:52.400724 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:51:52.401658 (+ 934us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:51:52.402262 (+ 604us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:51:52.403053 (+ 791us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:51:52.403262 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:51:52.403343 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:51:52.404783 (+ 1440us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:51:52.404787 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:51:52.404791 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:51:52.404840 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:51:52.406142 (+ 1302us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:51:52.406529 (+ 387us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:51:52.406531 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:51:52.406532 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:51:52.406606 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:51:52.406849 (+ 243us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:51:52.406851 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:51:52.406853 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:51:52.407061 (+ 208us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:51:52.407163 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0811 02:51:52.407399 (+ 236us) server_negotiation.cc:299] Negotiation successful
0811 02:51:52.407495 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260811 02:51:52.412523 13520 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39690:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 02:51:52.415678 13520 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 02:51:52.430141 14076 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:51:52.424469 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.138.195:42587 (local address 127.0.0.1:47182)
0811 02:51:52.425101 (+ 632us) negotiation.cc:107] Waiting for socket to connect
0811 02:51:52.425118 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 02:51:52.425251 (+ 133us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:51:52.426129 (+ 878us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:51:52.426135 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:51:52.426155 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 02:51:52.426265 (+ 110us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:51:52.426273 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:51:52.427299 (+ 1026us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:51:52.427311 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:51:52.428477 (+ 1166us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:51:52.428489 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:51:52.429550 (+ 1061us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:51:52.429554 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:51:52.429660 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:51:52.429696 (+ 36us) client_negotiation.cc:769] Sending connection context
0811 02:51:52.429857 (+ 161us) client_negotiation.cc:240] Negotiation successful
0811 02:51:52.429986 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":568,"mutex_wait_us":1,"thread_start_us":111,"threads_started":1}
I20260811 02:51:52.430154 14077 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:51:52.424990 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47182 (local address 127.12.138.195:42587)
0811 02:51:52.425441 (+ 451us) server_negotiation.cc:206] Beginning negotiation
0811 02:51:52.425461 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0811 02:51:52.425487 (+ 26us) server_negotiation.cc:407] Connection header received
0811 02:51:52.425567 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:51:52.425572 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:51:52.425786 (+ 214us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:51:52.425944 (+ 158us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 02:51:52.426401 (+ 457us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:51:52.427159 (+ 758us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:51:52.428619 (+ 1460us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:51:52.429427 (+ 808us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:51:52.429565 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:51:52.429704 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0811 02:51:52.429897 (+ 193us) server_negotiation.cc:299] Negotiation successful
0811 02:51:52.429989 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":353,"thread_start_us":107,"threads_started":1}
I20260811 02:51:52.430460 14075 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:51:52.424582 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.138.193:35251 (local address 127.0.0.1:33020)
0811 02:51:52.425294 (+ 712us) negotiation.cc:107] Waiting for socket to connect
0811 02:51:52.425308 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 02:51:52.425387 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:51:52.426056 (+ 669us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:51:52.426058 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:51:52.426070 (+ 12us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 02:51:52.426253 (+ 183us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:51:52.426261 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:51:52.427912 (+ 1651us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:51:52.427915 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:51:52.428955 (+ 1040us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:51:52.428966 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:51:52.430049 (+ 1083us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:51:52.430056 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:51:52.430150 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:51:52.430182 (+ 32us) client_negotiation.cc:769] Sending connection context
0811 02:51:52.430233 (+ 51us) client_negotiation.cc:240] Negotiation successful
0811 02:51:52.430325 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":647,"mutex_wait_us":27,"thread_start_us":76,"threads_started":1}
I20260811 02:51:52.430796 14073 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:51:52.424185 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.138.194:41449 (local address 127.0.0.1:57290)
0811 02:51:52.424921 (+ 736us) negotiation.cc:107] Waiting for socket to connect
0811 02:51:52.424972 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0811 02:51:52.425133 (+ 161us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:51:52.426029 (+ 896us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:51:52.426038 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:51:52.426065 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 02:51:52.426390 (+ 325us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:51:52.426399 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:51:52.427869 (+ 1470us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:51:52.427872 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:51:52.429045 (+ 1173us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:51:52.429055 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:51:52.430084 (+ 1029us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:51:52.430088 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:51:52.430165 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:51:52.430183 (+ 18us) client_negotiation.cc:769] Sending connection context
0811 02:51:52.430244 (+ 61us) client_negotiation.cc:240] Negotiation successful
0811 02:51:52.430666 (+ 422us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":551,"mutex_wait_us":562,"thread_start_us":93,"threads_started":1}
I20260811 02:51:52.430850 14074 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:51:52.424396 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57290 (local address 127.12.138.194:41449)
0811 02:51:52.425526 (+ 1130us) server_negotiation.cc:206] Beginning negotiation
0811 02:51:52.425539 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0811 02:51:52.425563 (+ 24us) server_negotiation.cc:407] Connection header received
0811 02:51:52.425641 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:51:52.425646 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:51:52.425838 (+ 192us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:51:52.426249 (+ 411us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 02:51:52.426512 (+ 263us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:51:52.427714 (+ 1202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:51:52.429191 (+ 1477us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:51:52.429955 (+ 764us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:51:52.430336 (+ 381us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:51:52.430459 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0811 02:51:52.430515 (+ 56us) server_negotiation.cc:299] Negotiation successful
0811 02:51:52.430710 (+ 195us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1022,"thread_start_us":135,"threads_started":1}
I20260811 02:51:52.431383 14078 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:51:52.425383 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33020 (local address 127.12.138.193:35251)
0811 02:51:52.425667 (+ 284us) server_negotiation.cc:206] Beginning negotiation
0811 02:51:52.425673 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 02:51:52.425691 (+ 18us) server_negotiation.cc:407] Connection header received
0811 02:51:52.425765 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:51:52.425769 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:51:52.425921 (+ 152us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:51:52.426540 (+ 619us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 02:51:52.426680 (+ 140us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:51:52.427767 (+ 1087us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:51:52.429062 (+ 1295us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:51:52.429892 (+ 830us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:51:52.430905 (+ 1013us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:51:52.431102 (+ 197us) server_negotiation.cc:1035] Waiting for connection context
0811 02:51:52.431149 (+ 47us) server_negotiation.cc:299] Negotiation successful
0811 02:51:52.431231 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":188,"thread_start_us":83,"threads_started":1}
I20260811 02:51:52.433960 13825 tablet_service.cc:1467] Processing CreateTablet for tablet 5e755752d9b74edc8d7747513400800e (DEFAULT_TABLE table=test-table [id=6ac26aaff17e4846833bd2012ce79130]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 02:51:52.433954 13667 tablet_service.cc:1467] Processing CreateTablet for tablet 5e755752d9b74edc8d7747513400800e (DEFAULT_TABLE table=test-table [id=6ac26aaff17e4846833bd2012ce79130]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 02:51:52.433954 13984 tablet_service.cc:1467] Processing CreateTablet for tablet 5e755752d9b74edc8d7747513400800e (DEFAULT_TABLE table=test-table [id=6ac26aaff17e4846833bd2012ce79130]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 02:51:52.435009 13825 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5e755752d9b74edc8d7747513400800e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 02:51:52.435029 13667 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5e755752d9b74edc8d7747513400800e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 02:51:52.435036 13984 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5e755752d9b74edc8d7747513400800e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 02:51:52.441183 14079 tablet_bootstrap.cc:492] T 5e755752d9b74edc8d7747513400800e P b1757237a2a243e492e295d2389f485d: Bootstrap starting.
I20260811 02:51:52.442590 14081 tablet_bootstrap.cc:492] T 5e755752d9b74edc8d7747513400800e P 6a07dfd0984340b2a25c3fed608b86a4: Bootstrap starting.
I20260811 02:51:52.444451 14079 tablet_bootstrap.cc:654] T 5e755752d9b74edc8d7747513400800e P b1757237a2a243e492e295d2389f485d: Neither blocks nor log segments found. Creating new log.
I20260811 02:51:52.444756 14081 tablet_bootstrap.cc:654] T 5e755752d9b74edc8d7747513400800e P 6a07dfd0984340b2a25c3fed608b86a4: Neither blocks nor log segments found. Creating new log.
I20260811 02:51:52.445442 14079 log.cc:824] T 5e755752d9b74edc8d7747513400800e P b1757237a2a243e492e295d2389f485d: Log is configured to *not* fsync() on all Append() calls
I20260811 02:51:52.445514 14082 tablet_bootstrap.cc:492] T 5e755752d9b74edc8d7747513400800e P afa0d92aa4bb434a8d219af54f8eff8d: Bootstrap starting.
I20260811 02:51:52.445580 14081 log.cc:824] T 5e755752d9b74edc8d7747513400800e P 6a07dfd0984340b2a25c3fed608b86a4: Log is configured to *not* fsync() on all Append() calls
I20260811 02:51:52.447347 14081 tablet_bootstrap.cc:492] T 5e755752d9b74edc8d7747513400800e P 6a07dfd0984340b2a25c3fed608b86a4: No bootstrap required, opened a new log
I20260811 02:51:52.447553 14081 ts_tablet_manager.cc:1397] T 5e755752d9b74edc8d7747513400800e P 6a07dfd0984340b2a25c3fed608b86a4: Time spent bootstrapping tablet: real 0.005s user 0.005s sys 0.000s
I20260811 02:51:52.447573 14082 tablet_bootstrap.cc:654] T 5e755752d9b74edc8d7747513400800e P afa0d92aa4bb434a8d219af54f8eff8d: Neither blocks nor log segments found. Creating new log.
I20260811 02:51:52.448767 14079 tablet_bootstrap.cc:492] T 5e755752d9b74edc8d7747513400800e P b1757237a2a243e492e295d2389f485d: No bootstrap required, opened a new log
I20260811 02:51:52.448859 14082 log.cc:824] T 5e755752d9b74edc8d7747513400800e P afa0d92aa4bb434a8d219af54f8eff8d: Log is configured to *not* fsync() on all Append() calls
I20260811 02:51:52.448947 14079 ts_tablet_manager.cc:1397] T 5e755752d9b74edc8d7747513400800e P b1757237a2a243e492e295d2389f485d: Time spent bootstrapping tablet: real 0.008s user 0.005s sys 0.000s
I20260811 02:51:52.450546 14082 tablet_bootstrap.cc:492] T 5e755752d9b74edc8d7747513400800e P afa0d92aa4bb434a8d219af54f8eff8d: No bootstrap required, opened a new log
I20260811 02:51:52.450714 14082 ts_tablet_manager.cc:1397] T 5e755752d9b74edc8d7747513400800e P afa0d92aa4bb434a8d219af54f8eff8d: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260811 02:51:52.452193 14081 raft_consensus.cc:348] T 5e755752d9b74edc8d7747513400800e P 6a07dfd0984340b2a25c3fed608b86a4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1757237a2a243e492e295d2389f485d" member_type: VOTER last_known_addr { host: "127.12.138.194" port: 41449 } } peers { permanent_uuid: "afa0d92aa4bb434a8d219af54f8eff8d" member_type: VOTER last_known_addr { host: "127.12.138.195" port: 42587 } } peers { permanent_uuid: "6a07dfd0984340b2a25c3fed608b86a4" member_type: VOTER last_known_addr { host: "127.12.138.193" port: 35251 } }
I20260811 02:51:52.452473 14081 raft_consensus.cc:374] T 5e755752d9b74edc8d7747513400800e P 6a07dfd0984340b2a25c3fed608b86a4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 02:51:52.452534 14081 raft_consensus.cc:729] T 5e755752d9b74edc8d7747513400800e P 6a07dfd0984340b2a25c3fed608b86a4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6a07dfd0984340b2a25c3fed608b86a4, State: Initialized, Role: FOLLOWER
I20260811 02:51:52.452804 14079 raft_consensus.cc:348] T 5e755752d9b74edc8d7747513400800e P b1757237a2a243e492e295d2389f485d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1757237a2a243e492e295d2389f485d" member_type: VOTER last_known_addr { host: "127.12.138.194" port: 41449 } } peers { permanent_uuid: "afa0d92aa4bb434a8d219af54f8eff8d" member_type: VOTER last_known_addr { host: "127.12.138.195" port: 42587 } } peers { permanent_uuid: "6a07dfd0984340b2a25c3fed608b86a4" member_type: VOTER last_known_addr { host: "127.12.138.193" port: 35251 } }
I20260811 02:51:52.452991 14081 consensus_queue.cc:260] T 5e755752d9b74edc8d7747513400800e P 6a07dfd0984340b2a25c3fed608b86a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1757237a2a243e492e295d2389f485d" member_type: VOTER last_known_addr { host: "127.12.138.194" port: 41449 } } peers { permanent_uuid: "afa0d92aa4bb434a8d219af54f8eff8d" member_type: VOTER last_known_addr { host: "127.12.138.195" port: 42587 } } peers { permanent_uuid: "6a07dfd0984340b2a25c3fed608b86a4" member_type: VOTER last_known_addr { host: "127.12.138.193" port: 35251 } }
I20260811 02:51:52.453068 14079 raft_consensus.cc:374] T 5e755752d9b74edc8d7747513400800e P b1757237a2a243e492e295d2389f485d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 02:51:52.453119 14079 raft_consensus.cc:729] T 5e755752d9b74edc8d7747513400800e P b1757237a2a243e492e295d2389f485d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b1757237a2a243e492e295d2389f485d, State: Initialized, Role: FOLLOWER
I20260811 02:51:52.453586 14079 consensus_queue.cc:260] T 5e755752d9b74edc8d7747513400800e P b1757237a2a243e492e295d2389f485d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1757237a2a243e492e295d2389f485d" member_type: VOTER last_known_addr { host: "127.12.138.194" port: 41449 } } peers { permanent_uuid: "afa0d92aa4bb434a8d219af54f8eff8d" member_type: VOTER last_known_addr { host: "127.12.138.195" port: 42587 } } peers { permanent_uuid: "6a07dfd0984340b2a25c3fed608b86a4" member_type: VOTER last_known_addr { host: "127.12.138.193" port: 35251 } }
I20260811 02:51:52.453842 14082 raft_consensus.cc:348] T 5e755752d9b74edc8d7747513400800e P afa0d92aa4bb434a8d219af54f8eff8d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1757237a2a243e492e295d2389f485d" member_type: VOTER last_known_addr { host: "127.12.138.194" port: 41449 } } peers { permanent_uuid: "afa0d92aa4bb434a8d219af54f8eff8d" member_type: VOTER last_known_addr { host: "127.12.138.195" port: 42587 } } peers { permanent_uuid: "6a07dfd0984340b2a25c3fed608b86a4" member_type: VOTER last_known_addr { host: "127.12.138.193" port: 35251 } }
I20260811 02:51:52.454044 14081 ts_tablet_manager.cc:1428] T 5e755752d9b74edc8d7747513400800e P 6a07dfd0984340b2a25c3fed608b86a4: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260811 02:51:52.454103 14082 raft_consensus.cc:374] T 5e755752d9b74edc8d7747513400800e P afa0d92aa4bb434a8d219af54f8eff8d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 02:51:52.454231 14082 raft_consensus.cc:729] T 5e755752d9b74edc8d7747513400800e P afa0d92aa4bb434a8d219af54f8eff8d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: afa0d92aa4bb434a8d219af54f8eff8d, State: Initialized, Role: FOLLOWER
I20260811 02:51:52.454391 13742 heartbeater.cc:503] Master 127.12.138.254:34899 was elected leader, sending a full tablet report...
I20260811 02:51:52.454499 14079 ts_tablet_manager.cc:1428] T 5e755752d9b74edc8d7747513400800e P b1757237a2a243e492e295d2389f485d: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260811 02:51:52.454612 14082 consensus_queue.cc:260] T 5e755752d9b74edc8d7747513400800e P afa0d92aa4bb434a8d219af54f8eff8d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1757237a2a243e492e295d2389f485d" member_type: VOTER last_known_addr { host: "127.12.138.194" port: 41449 } } peers { permanent_uuid: "afa0d92aa4bb434a8d219af54f8eff8d" member_type: VOTER last_known_addr { host: "127.12.138.195" port: 42587 } } peers { permanent_uuid: "6a07dfd0984340b2a25c3fed608b86a4" member_type: VOTER last_known_addr { host: "127.12.138.193" port: 35251 } }
I20260811 02:51:52.454893 13903 heartbeater.cc:503] Master 127.12.138.254:34899 was elected leader, sending a full tablet report...
I20260811 02:51:52.455528 14082 ts_tablet_manager.cc:1428] T 5e755752d9b74edc8d7747513400800e P afa0d92aa4bb434a8d219af54f8eff8d: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260811 02:51:52.455734 14059 heartbeater.cc:503] Master 127.12.138.254:34899 was elected leader, sending a full tablet report...
I20260811 02:51:52.522259 14087 raft_consensus.cc:482] T 5e755752d9b74edc8d7747513400800e P b1757237a2a243e492e295d2389f485d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 02:51:52.522528 14087 raft_consensus.cc:504] T 5e755752d9b74edc8d7747513400800e P b1757237a2a243e492e295d2389f485d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1757237a2a243e492e295d2389f485d" member_type: VOTER last_known_addr { host: "127.12.138.194" port: 41449 } } peers { permanent_uuid: "afa0d92aa4bb434a8d219af54f8eff8d" member_type: VOTER last_known_addr { host: "127.12.138.195" port: 42587 } } peers { permanent_uuid: "6a07dfd0984340b2a25c3fed608b86a4" member_type: VOTER last_known_addr { host: "127.12.138.193" port: 35251 } }
I20260811 02:51:52.523864 14087 leader_election.cc:288] T 5e755752d9b74edc8d7747513400800e P b1757237a2a243e492e295d2389f485d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers afa0d92aa4bb434a8d219af54f8eff8d (127.12.138.195:42587), 6a07dfd0984340b2a25c3fed608b86a4 (127.12.138.193:35251)
I20260811 02:51:52.527819 13906 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:51:52.524217 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.138.193:35251 (local address 127.12.138.194:45373)
0811 02:51:52.524434 (+ 217us) negotiation.cc:107] Waiting for socket to connect
0811 02:51:52.524451 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 02:51:52.524560 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:51:52.524869 (+ 309us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:51:52.524872 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:51:52.524891 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 02:51:52.525017 (+ 126us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:51:52.525022 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:51:52.525816 (+ 794us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:51:52.525821 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:51:52.526635 (+ 814us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:51:52.526654 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:51:52.527352 (+ 698us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:51:52.527356 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:51:52.527477 (+ 121us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:51:52.527502 (+ 25us) client_negotiation.cc:769] Sending connection context
0811 02:51:52.527597 (+ 95us) client_negotiation.cc:240] Negotiation successful
0811 02:51:52.527679 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":127,"mutex_wait_us":2}
I20260811 02:51:52.527820 14078 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:51:52.524375 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.138.194:45373 (local address 127.12.138.193:35251)
0811 02:51:52.524507 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0811 02:51:52.524512 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 02:51:52.524530 (+ 18us) server_negotiation.cc:407] Connection header received
0811 02:51:52.524709 (+ 179us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:51:52.524718 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:51:52.524762 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:51:52.524854 (+ 92us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 02:51:52.525135 (+ 281us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:51:52.525693 (+ 558us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:51:52.526783 (+ 1090us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:51:52.527220 (+ 437us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:51:52.527358 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:51:52.527414 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0811 02:51:52.527616 (+ 202us) server_negotiation.cc:299] Negotiation successful
0811 02:51:52.527688 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260811 02:51:52.528532 13691 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5e755752d9b74edc8d7747513400800e" candidate_uuid: "b1757237a2a243e492e295d2389f485d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6a07dfd0984340b2a25c3fed608b86a4" is_pre_election: true
I20260811 02:51:52.528704 14090 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:51:52.524240 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.138.195:42587 (local address 127.12.138.194:52189)
0811 02:51:52.524721 (+ 481us) negotiation.cc:107] Waiting for socket to connect
0811 02:51:52.524733 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 02:51:52.524883 (+ 150us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:51:52.525218 (+ 335us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:51:52.525222 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:51:52.525240 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 02:51:52.525382 (+ 142us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:51:52.525388 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:51:52.526280 (+ 892us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:51:52.526286 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:51:52.527330 (+ 1044us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:51:52.527341 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:51:52.528202 (+ 861us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:51:52.528206 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:51:52.528332 (+ 126us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:51:52.528357 (+ 25us) client_negotiation.cc:769] Sending connection context
0811 02:51:52.528451 (+ 94us) client_negotiation.cc:240] Negotiation successful
0811 02:51:52.528523 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":421,"thread_start_us":154,"threads_started":1}
I20260811 02:51:52.528721 14077 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:51:52.524341 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.138.194:52189 (local address 127.12.138.195:42587)
0811 02:51:52.524493 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0811 02:51:52.524498 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 02:51:52.524849 (+ 351us) server_negotiation.cc:407] Connection header received
0811 02:51:52.525020 (+ 171us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:51:52.525029 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:51:52.525074 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:51:52.525196 (+ 122us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 02:51:52.525508 (+ 312us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:51:52.526151 (+ 643us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:51:52.527474 (+ 1323us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:51:52.528015 (+ 541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:51:52.528163 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:51:52.528223 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0811 02:51:52.528500 (+ 277us) server_negotiation.cc:299] Negotiation successful
0811 02:51:52.528591 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260811 02:51:52.528903 13691 raft_consensus.cc:2454] T 5e755752d9b74edc8d7747513400800e P 6a07dfd0984340b2a25c3fed608b86a4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b1757237a2a243e492e295d2389f485d in term 0.
I20260811 02:51:52.529464 13772 leader_election.cc:302] T 5e755752d9b74edc8d7747513400800e P b1757237a2a243e492e295d2389f485d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a07dfd0984340b2a25c3fed608b86a4, b1757237a2a243e492e295d2389f485d; no voters:
I20260811 02:51:52.529508 14009 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5e755752d9b74edc8d7747513400800e" candidate_uuid: "b1757237a2a243e492e295d2389f485d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "afa0d92aa4bb434a8d219af54f8eff8d" is_pre_election: true
I20260811 02:51:52.529785 14009 raft_consensus.cc:2454] T 5e755752d9b74edc8d7747513400800e P afa0d92aa4bb434a8d219af54f8eff8d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b1757237a2a243e492e295d2389f485d in term 0.
I20260811 02:51:52.529799 14087 raft_consensus.cc:2781] T 5e755752d9b74edc8d7747513400800e P b1757237a2a243e492e295d2389f485d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 02:51:52.529923 14087 raft_consensus.cc:482] T 5e755752d9b74edc8d7747513400800e P b1757237a2a243e492e295d2389f485d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 02:51:52.530012 14087 raft_consensus.cc:3037] T 5e755752d9b74edc8d7747513400800e P b1757237a2a243e492e295d2389f485d [term 0 FOLLOWER]: Advancing to term 1
I20260811 02:51:52.531134 14087 raft_consensus.cc:504] T 5e755752d9b74edc8d7747513400800e P b1757237a2a243e492e295d2389f485d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1757237a2a243e492e295d2389f485d" member_type: VOTER last_known_addr { host: "127.12.138.194" port: 41449 } } peers { permanent_uuid: "afa0d92aa4bb434a8d219af54f8eff8d" member_type: VOTER last_known_addr { host: "127.12.138.195" port: 42587 } } peers { permanent_uuid: "6a07dfd0984340b2a25c3fed608b86a4" member_type: VOTER last_known_addr { host: "127.12.138.193" port: 35251 } }
I20260811 02:51:52.531447 14087 leader_election.cc:288] T 5e755752d9b74edc8d7747513400800e P b1757237a2a243e492e295d2389f485d [CANDIDATE]: Term 1 election: Requested vote from peers afa0d92aa4bb434a8d219af54f8eff8d (127.12.138.195:42587), 6a07dfd0984340b2a25c3fed608b86a4 (127.12.138.193:35251)
I20260811 02:51:52.531906 13691 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5e755752d9b74edc8d7747513400800e" candidate_uuid: "b1757237a2a243e492e295d2389f485d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6a07dfd0984340b2a25c3fed608b86a4"
I20260811 02:51:52.532014 14009 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5e755752d9b74edc8d7747513400800e" candidate_uuid: "b1757237a2a243e492e295d2389f485d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "afa0d92aa4bb434a8d219af54f8eff8d"
I20260811 02:51:52.532122 13691 raft_consensus.cc:3037] T 5e755752d9b74edc8d7747513400800e P 6a07dfd0984340b2a25c3fed608b86a4 [term 0 FOLLOWER]: Advancing to term 1
I20260811 02:51:52.532132 14009 raft_consensus.cc:3037] T 5e755752d9b74edc8d7747513400800e P afa0d92aa4bb434a8d219af54f8eff8d [term 0 FOLLOWER]: Advancing to term 1
I20260811 02:51:52.533716 13691 raft_consensus.cc:2454] T 5e755752d9b74edc8d7747513400800e P 6a07dfd0984340b2a25c3fed608b86a4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b1757237a2a243e492e295d2389f485d in term 1.
I20260811 02:51:52.533712 14009 raft_consensus.cc:2454] T 5e755752d9b74edc8d7747513400800e P afa0d92aa4bb434a8d219af54f8eff8d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b1757237a2a243e492e295d2389f485d in term 1.
I20260811 02:51:52.534087 13771 leader_election.cc:302] T 5e755752d9b74edc8d7747513400800e P b1757237a2a243e492e295d2389f485d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: afa0d92aa4bb434a8d219af54f8eff8d, b1757237a2a243e492e295d2389f485d; no voters:
I20260811 02:51:52.534300 14087 raft_consensus.cc:2781] T 5e755752d9b74edc8d7747513400800e P b1757237a2a243e492e295d2389f485d [term 1 FOLLOWER]: Leader election won for term 1
I20260811 02:51:52.534544 14087 raft_consensus.cc:686] T 5e755752d9b74edc8d7747513400800e P b1757237a2a243e492e295d2389f485d [term 1 LEADER]: Becoming Leader. State: Replica: b1757237a2a243e492e295d2389f485d, State: Running, Role: LEADER
I20260811 02:51:52.534909 14087 consensus_queue.cc:237] T 5e755752d9b74edc8d7747513400800e P b1757237a2a243e492e295d2389f485d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1757237a2a243e492e295d2389f485d" member_type: VOTER last_known_addr { host: "127.12.138.194" port: 41449 } } peers { permanent_uuid: "afa0d92aa4bb434a8d219af54f8eff8d" member_type: VOTER last_known_addr { host: "127.12.138.195" port: 42587 } } peers { permanent_uuid: "6a07dfd0984340b2a25c3fed608b86a4" member_type: VOTER last_known_addr { host: "127.12.138.193" port: 35251 } }
I20260811 02:51:52.538321 13515 catalog_manager.cc:5495] T 5e755752d9b74edc8d7747513400800e P b1757237a2a243e492e295d2389f485d reported cstate change: term changed from 0 to 1, leader changed from <none> to b1757237a2a243e492e295d2389f485d (127.12.138.194). New cstate: current_term: 1 leader_uuid: "b1757237a2a243e492e295d2389f485d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1757237a2a243e492e295d2389f485d" member_type: VOTER last_known_addr { host: "127.12.138.194" port: 41449 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "afa0d92aa4bb434a8d219af54f8eff8d" member_type: VOTER last_known_addr { host: "127.12.138.195" port: 42587 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6a07dfd0984340b2a25c3fed608b86a4" member_type: VOTER last_known_addr { host: "127.12.138.193" port: 35251 } health_report { overall_health: UNKNOWN } } }
Aug 11 02:51:52 dist-test-slave-9gcw krb5kdc[13467](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416711, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.12.138.193@KRBTEST.COM
I20260811 02:51:52.564756 14078 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:51:52.555492 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33022 (local address 127.12.138.193:35251)
0811 02:51:52.555647 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0811 02:51:52.555652 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 02:51:52.555859 (+ 207us) server_negotiation.cc:407] Connection header received
0811 02:51:52.555968 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:51:52.555971 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:51:52.556015 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:51:52.556086 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:51:52.556749 (+ 663us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:51:52.557322 (+ 573us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:51:52.558159 (+ 837us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:51:52.558378 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:51:52.558457 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:51:52.561027 (+ 2570us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:51:52.561040 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:51:52.561088 (+ 48us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:51:52.561116 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:51:52.562930 (+ 1814us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:51:52.563357 (+ 427us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:51:52.563363 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:51:52.563364 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:51:52.563454 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:51:52.563708 (+ 254us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:51:52.563711 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:51:52.563713 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:51:52.563985 (+ 272us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:51:52.564190 (+ 205us) server_negotiation.cc:1035] Waiting for connection context
0811 02:51:52.564395 (+ 205us) server_negotiation.cc:299] Negotiation successful
0811 02:51:52.564545 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
Aug 11 02:51:52 dist-test-slave-9gcw krb5kdc[13467](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416712, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 11 02:51:52 dist-test-slave-9gcw krb5kdc[13467](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416712, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.12.138.193@KRBTEST.COM
I20260811 02:51:52.596505 14078 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:51:52.587375 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33034 (local address 127.12.138.193:35251)
0811 02:51:52.587560 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0811 02:51:52.587566 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 02:51:52.587713 (+ 147us) server_negotiation.cc:407] Connection header received
0811 02:51:52.587945 (+ 232us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:51:52.587950 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:51:52.588000 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:51:52.588088 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:51:52.588797 (+ 709us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:51:52.589425 (+ 628us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:51:52.590273 (+ 848us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:51:52.590511 (+ 238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:51:52.590620 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:51:52.593288 (+ 2668us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:51:52.593292 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:51:52.593294 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:51:52.593318 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:51:52.594864 (+ 1546us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:51:52.595319 (+ 455us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:51:52.595321 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:51:52.595322 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:51:52.595412 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:51:52.595666 (+ 254us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:51:52.595668 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:51:52.595669 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:51:52.595870 (+ 201us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:51:52.595974 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0811 02:51:52.596275 (+ 301us) server_negotiation.cc:299] Negotiation successful
0811 02:51:52.596381 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":83}
Aug 11 02:51:52 dist-test-slave-9gcw krb5kdc[13467](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416712, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 11 02:51:52 dist-test-slave-9gcw krb5kdc[13467](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416712, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.12.138.193@KRBTEST.COM
W20260811 02:51:52.624527 14060 tablet.cc:2369] T 5e755752d9b74edc8d7747513400800e P afa0d92aa4bb434a8d219af54f8eff8d: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 02:51:52.627548 14078 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:51:52.618287 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33050 (local address 127.12.138.193:35251)
0811 02:51:52.618473 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0811 02:51:52.618478 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 02:51:52.618629 (+ 151us) server_negotiation.cc:407] Connection header received
0811 02:51:52.618817 (+ 188us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:51:52.618821 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:51:52.618864 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:51:52.618963 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:51:52.619691 (+ 728us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:51:52.620204 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:51:52.621078 (+ 874us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:51:52.621368 (+ 290us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:51:52.621490 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:51:52.624242 (+ 2752us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:51:52.624246 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:51:52.624248 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:51:52.624275 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:51:52.625873 (+ 1598us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:51:52.626352 (+ 479us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:51:52.626355 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:51:52.626356 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:51:52.626430 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:51:52.626780 (+ 350us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:51:52.626782 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:51:52.626783 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:51:52.626957 (+ 174us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:51:52.627056 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0811 02:51:52.627313 (+ 257us) server_negotiation.cc:299] Negotiation successful
0811 02:51:52.627418 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82}
W20260811 02:51:52.629019 13643 tablet_service.cc:2908] Rejecting scan request for tablet 5e755752d9b74edc8d7747513400800e: Uninitialized: clean time has not yet been initialized
I20260811 02:51:52.631325 12843 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDawEwS/build/debug/bin/kudu with pid 13589
W20260811 02:51:52.634519 13904 tablet.cc:2369] T 5e755752d9b74edc8d7747513400800e P b1757237a2a243e492e295d2389f485d: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 02:51:52.643052 12843 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDawEwS/build/debug/bin/kudu with pid 13750
I20260811 02:51:52.658541 12843 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDawEwS/build/debug/bin/kudu with pid 13909
I20260811 02:51:52.665576 12843 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDawEwS/build/debug/bin/kudu with pid 13481
2026-08-11T02:51:52Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnChecksum (2545 ms)
[ RUN ] SecurityITest.SmokeTestAsAuthorizedUser
Loading random data
Initializing database '/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786416707705494-12843-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 02:51:52 dist-test-slave-9gcw krb5kdc[14106](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 02:51:52 dist-test-slave-9gcw krb5kdc[14106](info): set up 2 sockets
Aug 11 02:51:52 dist-test-slave-9gcw krb5kdc[14106](info): commencing operation
krb5kdc: starting...
W20260811 02:51:53.841444 12843 mini_kdc.cc:121] Time spent starting KDC: real 1.149s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786416707705494-12843-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786416707705494-12843-0/krb5kdc/test-user.keytab.
Aug 11 02:51:53 dist-test-slave-9gcw krb5kdc[14106](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416713, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T02:51:53Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T02:51:53Z Disabled control of system clock
WARNING: no policy specified for kudu/127.12.138.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.138.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.138.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786416707705494-12843-0/krb5kdc/kudu_127.12.138.254.keytab.
Entry for principal kudu/127.12.138.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786416707705494-12843-0/krb5kdc/kudu_127.12.138.254.keytab.
I20260811 02:51:53.989871 12843 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDawEwS/build/debug/bin/kudu
/tmp/dist-test-taskDawEwS/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786416707705494-12843-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786416707705494-12843-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786416707705494-12843-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786416707705494-12843-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.138.254:45929
--webserver_interface=127.12.138.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786416707705494-12843-0/krb5kdc/kudu_127.12.138.254.keytab
--principal=kudu/127.12.138.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.138.212:39955
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.12.138.254:45929
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786416707705494-12843-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786416707705494-12843-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786416707705494-12843-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 02:51:54.118227 14122 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 02:51:54.118571 14122 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 02:51:54.118654 14122 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 02:51:54.122829 14122 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 02:51:54.122951 14122 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260811 02:51:54.122987 14122 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 02:51:54.123031 14122 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 02:51:54.123071 14122 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 02:51:54.123109 14122 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 02:51:54.128650 14122 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.138.212:39955
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786416707705494-12843-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786416707705494-12843-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.12.138.254:45929
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786416707705494-12843-0/krb5kdc/kudu_127.12.138.254.keytab
--principal=kudu/127.12.138.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.138.254:45929
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786416707705494-12843-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.12.138.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14122
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786416707705494-12843-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 02:09:49 UTC on ccycloud.quasar-fsnqqc.root.comops.site
build id 4579570
I20260811 02:51:54.129927 14122 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 02:51:54.131018 14122 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 02:51:54.140322 14127 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:51:54.140446 14122 server_base.cc:1034] running on GCE node
W20260811 02:51:54.140331 14130 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:51:54.140347 14128 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:51:54.141445 14122 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 02:51:54.142953 14122 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 02:51:54.144151 14122 hybrid_clock.cc:648] HybridClock initialized: now 1786416714144119 us; error 54 us; skew 500 ppm
Aug 11 02:51:54 dist-test-slave-9gcw krb5kdc[14106](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416714, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.138.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 02:51:54.150652 14122 init.cc:377] Logged in from keytab as kudu/127.12.138.254@KRBTEST.COM (short username kudu)
I20260811 02:51:54.151746 14122 webserver.cc:460] Webserver started at http://127.12.138.254:46117/ using document root <none> and password file <none>
I20260811 02:51:54.152354 14122 fs_manager.cc:359] Metadata directory not provided
I20260811 02:51:54.152419 14122 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 02:51:54.152655 14122 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 02:51:54.154902 14122 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786416707705494-12843-0/minicluster-data/master-0/data/instance:
uuid: "7514043a26cd4e49b389e3b8af39c2ed"
format_stamp: "Formatted at 2026-08-11 02:51:54 on dist-test-slave-9gcw"
server_key: "b85d73bcc8250a6327024094d841bb0f"
server_key_iv: "c55a106693c5136f479029233019eed0"
server_key_version: "encryptionkey@0"
I20260811 02:51:54.155514 14122 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786416707705494-12843-0/minicluster-data/master-0/wal/instance:
uuid: "7514043a26cd4e49b389e3b8af39c2ed"
format_stamp: "Formatted at 2026-08-11 02:51:54 on dist-test-slave-9gcw"
server_key: "b85d73bcc8250a6327024094d841bb0f"
server_key_iv: "c55a106693c5136f479029233019eed0"
server_key_version: "encryptionkey@0"
I20260811 02:51:54.159584 14122 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.001s
I20260811 02:51:54.162267 14137 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 02:51:54.163654 14122 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 02:51:54.163761 14122 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786416707705494-12843-0/minicluster-data/master-0/data,/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786416707705494-12843-0/minicluster-data/master-0/wal
uuid: "7514043a26cd4e49b389e3b8af39c2ed"
format_stamp: "Formatted at 2026-08-11 02:51:54 on dist-test-slave-9gcw"
server_key: "b85d73bcc8250a6327024094d841bb0f"
server_key_iv: "c55a106693c5136f479029233019eed0"
server_key_version: "encryptionkey@0"
I20260811 02:51:54.163909 14122 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786416707705494-12843-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786416707705494-12843-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786416707705494-12843-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 02:51:54.198446 14122 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 02:51:54.201933 14122 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 02:51:54.202175 14122 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 02:51:54.211308 14122 rpc_server.cc:307] RPC server started. Bound to: 127.12.138.254:45929
I20260811 02:51:54.211323 14212 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.138.254:45929 every 8 connection(s)
I20260811 02:51:54.212345 14122 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786416707705494-12843-0/minicluster-data/master-0/data/info.pb
I20260811 02:51:54.215353 14213 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 02:51:54.219053 12843 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDawEwS/build/debug/bin/kudu as pid 14122
I20260811 02:51:54.219173 12843 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786416707705494-12843-0/minicluster-data/master-0/wal/instance
I20260811 02:51:54.219451 12843 external_mini_cluster.cc:1428] Setting key 92775996e20f20490d286abef26b9125
I20260811 02:51:54.221460 14213 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7514043a26cd4e49b389e3b8af39c2ed: Bootstrap starting.
I20260811 02:51:54.223644 14213 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7514043a26cd4e49b389e3b8af39c2ed: Neither blocks nor log segments found. Creating new log.
I20260811 02:51:54.224386 14213 log.cc:824] T 00000000000000000000000000000000 P 7514043a26cd4e49b389e3b8af39c2ed: Log is configured to *not* fsync() on all Append() calls
I20260811 02:51:54.226311 14213 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7514043a26cd4e49b389e3b8af39c2ed: No bootstrap required, opened a new log
Aug 11 02:51:54 dist-test-slave-9gcw krb5kdc[14106](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416713, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.12.138.254@KRBTEST.COM
I20260811 02:51:54.230157 14213 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7514043a26cd4e49b389e3b8af39c2ed [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7514043a26cd4e49b389e3b8af39c2ed" member_type: VOTER last_known_addr { host: "127.12.138.254" port: 45929 } }
I20260811 02:51:54.230412 14213 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7514043a26cd4e49b389e3b8af39c2ed [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 02:51:54.230499 14213 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7514043a26cd4e49b389e3b8af39c2ed [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7514043a26cd4e49b389e3b8af39c2ed, State: Initialized, Role: FOLLOWER
I20260811 02:51:54.230991 14213 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7514043a26cd4e49b389e3b8af39c2ed [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7514043a26cd4e49b389e3b8af39c2ed" member_type: VOTER last_known_addr { host: "127.12.138.254" port: 45929 } }
I20260811 02:51:54.231150 14213 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7514043a26cd4e49b389e3b8af39c2ed [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 02:51:54.231240 14213 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7514043a26cd4e49b389e3b8af39c2ed [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 02:51:54.231339 14213 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7514043a26cd4e49b389e3b8af39c2ed [term 0 FOLLOWER]: Advancing to term 1
I20260811 02:51:54.232242 14213 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7514043a26cd4e49b389e3b8af39c2ed [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7514043a26cd4e49b389e3b8af39c2ed" member_type: VOTER last_known_addr { host: "127.12.138.254" port: 45929 } }
I20260811 02:51:54.232605 14213 leader_election.cc:302] T 00000000000000000000000000000000 P 7514043a26cd4e49b389e3b8af39c2ed [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7514043a26cd4e49b389e3b8af39c2ed; no voters:
I20260811 02:51:54.232995 14213 leader_election.cc:288] T 00000000000000000000000000000000 P 7514043a26cd4e49b389e3b8af39c2ed [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 02:51:54.233186 14218 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7514043a26cd4e49b389e3b8af39c2ed [term 1 FOLLOWER]: Leader election won for term 1
I20260811 02:51:54.233422 14218 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7514043a26cd4e49b389e3b8af39c2ed [term 1 LEADER]: Becoming Leader. State: Replica: 7514043a26cd4e49b389e3b8af39c2ed, State: Running, Role: LEADER
I20260811 02:51:54.233727 14218 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7514043a26cd4e49b389e3b8af39c2ed [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7514043a26cd4e49b389e3b8af39c2ed" member_type: VOTER last_known_addr { host: "127.12.138.254" port: 45929 } }
I20260811 02:51:54.234279 14213 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7514043a26cd4e49b389e3b8af39c2ed [sys.catalog]: configured and running, proceeding with master startup.
I20260811 02:51:54.234524 14216 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:51:54.220926 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59742 (local address 127.12.138.254:45929)
0811 02:51:54.221406 (+ 480us) server_negotiation.cc:206] Beginning negotiation
0811 02:51:54.221421 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0811 02:51:54.221460 (+ 39us) server_negotiation.cc:407] Connection header received
0811 02:51:54.222139 (+ 679us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:51:54.222155 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:51:54.222492 (+ 337us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:51:54.222830 (+ 338us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:51:54.223660 (+ 830us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:51:54.224802 (+ 1142us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:51:54.225723 (+ 921us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:51:54.226205 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:51:54.226387 (+ 182us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:51:54.228973 (+ 2586us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:51:54.228978 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:51:54.228992 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:51:54.229019 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:51:54.230910 (+ 1891us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:51:54.231440 (+ 530us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:51:54.231448 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:51:54.231454 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:51:54.231583 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:51:54.231885 (+ 302us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:51:54.231888 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:51:54.231890 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:51:54.232301 (+ 411us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:51:54.232544 (+ 243us) server_negotiation.cc:1035] Waiting for connection context
0811 02:51:54.232860 (+ 316us) server_negotiation.cc:299] Negotiation successful
0811 02:51:54.233403 (+ 543us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":331,"thread_start_us":160,"threads_started":1}
I20260811 02:51:54.234932 14218 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7514043a26cd4e49b389e3b8af39c2ed [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7514043a26cd4e49b389e3b8af39c2ed. Latest consensus state: current_term: 1 leader_uuid: "7514043a26cd4e49b389e3b8af39c2ed" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7514043a26cd4e49b389e3b8af39c2ed" member_type: VOTER last_known_addr { host: "127.12.138.254" port: 45929 } } }
I20260811 02:51:54.235051 14218 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7514043a26cd4e49b389e3b8af39c2ed [sys.catalog]: This master's current role is: LEADER
I20260811 02:51:54.235526 14219 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7514043a26cd4e49b389e3b8af39c2ed [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7514043a26cd4e49b389e3b8af39c2ed" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7514043a26cd4e49b389e3b8af39c2ed" member_type: VOTER last_known_addr { host: "127.12.138.254" port: 45929 } } }
I20260811 02:51:54.235647 14219 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7514043a26cd4e49b389e3b8af39c2ed [sys.catalog]: This master's current role is: LEADER
W20260811 02:51:54.239394 14237 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 7514043a26cd4e49b389e3b8af39c2ed: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 02:51:54.239477 14237 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 02:51:54.239533 14226 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 02:51:54.240404 14226 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 02:51:54.246059 14226 catalog_manager.cc:1347] Generated new cluster ID: 17ba4f128c824e3fb4f18c13f4174292
I20260811 02:51:54.246130 14226 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 02:51:54.262487 14226 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 02:51:54.263438 14226 catalog_manager.cc:1504] Loading token signing keys...
I20260811 02:51:54.268800 14226 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7514043a26cd4e49b389e3b8af39c2ed: Generated new TSK 0
I20260811 02:51:54.269618 14226 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.12.138.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.138.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.138.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786416707705494-12843-0/krb5kdc/kudu_127.12.138.193.keytab.
Entry for principal kudu/127.12.138.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786416707705494-12843-0/krb5kdc/kudu_127.12.138.193.keytab.
I20260811 02:51:54.306875 12843 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDawEwS/build/debug/bin/kudu
/tmp/dist-test-taskDawEwS/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786416707705494-12843-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786416707705494-12843-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786416707705494-12843-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786416707705494-12843-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.138.193:0
--local_ip_for_outbound_sockets=127.12.138.193
--webserver_interface=127.12.138.193
--webserver_port=0
--tserver_master_addrs=127.12.138.254:45929
--keytab_file=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786416707705494-12843-0/krb5kdc/kudu_127.12.138.193.keytab
--principal=kudu/127.12.138.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.138.212:39955
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786416707705494-12843-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786416707705494-12843-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786416707705494-12843-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 02:51:54.425755 14245 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 02:51:54.426030 14245 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 02:51:54.426079 14245 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 02:51:54.430195 14245 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 02:51:54.430265 14245 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 02:51:54.430308 14245 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260811 02:51:54.430405 14245 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.138.193
I20260811 02:51:54.435993 14245 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.138.212:39955
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786416707705494-12843-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786416707705494-12843-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786416707705494-12843-0/krb5kdc/kudu_127.12.138.193.keytab
--principal=kudu/127.12.138.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.138.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786416707705494-12843-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.12.138.193
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.12.138.254:45929
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14245
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.138.193
--log_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786416707705494-12843-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 02:09:49 UTC on ccycloud.quasar-fsnqqc.root.comops.site
build id 4579570
I20260811 02:51:54.437256 14245 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 02:51:54.438371 14245 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 02:51:54.449301 14254 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:51:54.449308 14251 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:51:54.449313 14252 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:51:54.449725 14245 server_base.cc:1034] running on GCE node
I20260811 02:51:54.450210 14245 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 02:51:54.450958 14245 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 02:51:54.452193 14245 hybrid_clock.cc:648] HybridClock initialized: now 1786416714452170 us; error 41 us; skew 500 ppm
Aug 11 02:51:54 dist-test-slave-9gcw krb5kdc[14106](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416714, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.138.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 02:51:54.455596 14245 init.cc:377] Logged in from keytab as kudu/127.12.138.193@KRBTEST.COM (short username kudu)
I20260811 02:51:54.456583 14245 webserver.cc:460] Webserver started at http://127.12.138.193:37887/ using document root <none> and password file <none>
I20260811 02:51:54.457315 14245 fs_manager.cc:359] Metadata directory not provided
I20260811 02:51:54.457427 14245 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 02:51:54.457700 14245 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 02:51:54.459980 14245 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786416707705494-12843-0/minicluster-data/ts-0/data/instance:
uuid: "c9b7177680454e789d9e860ba91be083"
format_stamp: "Formatted at 2026-08-11 02:51:54 on dist-test-slave-9gcw"
server_key: "bace140b728b8cef9b8cb420138c1396"
server_key_iv: "63bdbe2de4d7f4477f58c5c2652dd055"
server_key_version: "encryptionkey@0"
I20260811 02:51:54.460629 14245 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786416707705494-12843-0/minicluster-data/ts-0/wal/instance:
uuid: "c9b7177680454e789d9e860ba91be083"
format_stamp: "Formatted at 2026-08-11 02:51:54 on dist-test-slave-9gcw"
server_key: "bace140b728b8cef9b8cb420138c1396"
server_key_iv: "63bdbe2de4d7f4477f58c5c2652dd055"
server_key_version: "encryptionkey@0"
I20260811 02:51:54.464972 14245 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.007s sys 0.000s
I20260811 02:51:54.467814 14262 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 02:51:54.469420 14245 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 02:51:54.469586 14245 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786416707705494-12843-0/minicluster-data/ts-0/data,/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786416707705494-12843-0/minicluster-data/ts-0/wal
uuid: "c9b7177680454e789d9e860ba91be083"
format_stamp: "Formatted at 2026-08-11 02:51:54 on dist-test-slave-9gcw"
server_key: "bace140b728b8cef9b8cb420138c1396"
server_key_iv: "63bdbe2de4d7f4477f58c5c2652dd055"
server_key_version: "encryptionkey@0"
I20260811 02:51:54.469720 14245 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786416707705494-12843-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786416707705494-12843-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786416707705494-12843-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 02:51:54.480566 14245 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 02:51:54.483989 14245 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 02:51:54.484241 14245 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 02:51:54.486058 14245 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 02:51:54.486151 14245 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 02:51:54.486227 14245 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 02:51:54.486276 14245 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 11 02:51:54 dist-test-slave-9gcw krb5kdc[14106](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416714, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.138.193@KRBTEST.COM for kudu/127.12.138.254@KRBTEST.COM
I20260811 02:51:54.498765 14245 rpc_server.cc:307] RPC server started. Bound to: 127.12.138.193:43377
I20260811 02:51:54.499126 14398 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.138.193:43377 every 8 connection(s)
I20260811 02:51:54.500005 14245 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786416707705494-12843-0/minicluster-data/ts-0/data/info.pb
I20260811 02:51:54.501922 14216 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:51:54.486769 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.138.193:34223 (local address 127.12.138.254:45929)
0811 02:51:54.486954 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0811 02:51:54.486960 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 02:51:54.488209 (+ 1249us) server_negotiation.cc:407] Connection header received
0811 02:51:54.489570 (+ 1361us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:51:54.489573 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:51:54.489637 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:51:54.489733 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:51:54.491485 (+ 1752us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:51:54.492334 (+ 849us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:51:54.494306 (+ 1972us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:51:54.494657 (+ 351us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:51:54.494778 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:51:54.497783 (+ 3005us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:51:54.497788 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:51:54.497791 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:51:54.497828 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:51:54.499684 (+ 1856us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:51:54.500236 (+ 552us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:51:54.500240 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:51:54.500242 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:51:54.500357 (+ 115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:51:54.500774 (+ 417us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:51:54.500778 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:51:54.500779 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:51:54.501046 (+ 267us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:51:54.501131 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0811 02:51:54.501678 (+ 547us) server_negotiation.cc:299] Negotiation successful
0811 02:51:54.501790 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260811 02:51:54.502730 14273 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:51:54.487599 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.138.254:45929 (local address 127.12.138.193:34223)
0811 02:51:54.488050 (+ 451us) negotiation.cc:107] Waiting for socket to connect
0811 02:51:54.488102 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0811 02:51:54.489341 (+ 1239us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:51:54.489990 (+ 649us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:51:54.490002 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:51:54.490529 (+ 527us) client_negotiation.cc:189] Negotiated authn=SASL
0811 02:51:54.491311 (+ 782us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:51:54.491338 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:51:54.492471 (+ 1133us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:51:54.492477 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:51:54.494133 (+ 1656us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:51:54.494149 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:51:54.494833 (+ 684us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:51:54.494838 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:51:54.495035 (+ 197us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:51:54.495743 (+ 708us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 02:51:54.495759 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0811 02:51:54.497462 (+ 1703us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 02:51:54.499805 (+ 2343us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:51:54.499810 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:51:54.499839 (+ 29us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:51:54.500097 (+ 258us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:51:54.500474 (+ 377us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:51:54.500478 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:51:54.500480 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:51:54.500650 (+ 170us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:51:54.501184 (+ 534us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 02:51:54.501190 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 02:51:54.501439 (+ 249us) client_negotiation.cc:769] Sending connection context
0811 02:51:54.501614 (+ 175us) client_negotiation.cc:240] Negotiation successful
0811 02:51:54.501875 (+ 261us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":250,"thread_start_us":137,"threads_started":1}
I20260811 02:51:54.504040 14399 heartbeater.cc:347] Connected to a master server at 127.12.138.254:45929
I20260811 02:51:54.504182 14399 heartbeater.cc:464] Registering TS with master...
I20260811 02:51:54.504747 14399 heartbeater.cc:511] Master 127.12.138.254:45929 requested a full tablet report, sending...
I20260811 02:51:54.505255 12843 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDawEwS/build/debug/bin/kudu as pid 14245
I20260811 02:51:54.505388 12843 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786416707705494-12843-0/minicluster-data/ts-0/wal/instance
I20260811 02:51:54.505748 12843 external_mini_cluster.cc:1428] Setting key 90e43e2158a1a6c5b1a69e0a39a639bc
I20260811 02:51:54.507575 14159 ts_manager.cc:194] Registered new tserver with Master: c9b7177680454e789d9e860ba91be083 (127.12.138.193:43377)
I20260811 02:51:54.509152 14159 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.12.138.193@KRBTEST.COM'} at 127.12.138.193:34223
I20260811 02:51:54.517041 14409 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:51:54.508125 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.138.254:45929 (local address 127.12.138.193:60073)
0811 02:51:54.508448 (+ 323us) negotiation.cc:107] Waiting for socket to connect
0811 02:51:54.508467 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0811 02:51:54.508584 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:51:54.508877 (+ 293us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:51:54.508881 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:51:54.509130 (+ 249us) client_negotiation.cc:189] Negotiated authn=SASL
0811 02:51:54.509330 (+ 200us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:51:54.509340 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:51:54.510337 (+ 997us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:51:54.510343 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:51:54.511359 (+ 1016us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:51:54.511371 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:51:54.511999 (+ 628us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:51:54.512002 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:51:54.512079 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:51:54.512736 (+ 657us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 02:51:54.512756 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0811 02:51:54.513122 (+ 366us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 02:51:54.515333 (+ 2211us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:51:54.515348 (+ 15us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:51:54.515349 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:51:54.515567 (+ 218us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:51:54.515905 (+ 338us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:51:54.515907 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:51:54.515908 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:51:54.515999 (+ 91us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:51:54.516493 (+ 494us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 02:51:54.516496 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 02:51:54.516582 (+ 86us) client_negotiation.cc:769] Sending connection context
0811 02:51:54.516756 (+ 174us) client_negotiation.cc:240] Negotiation successful
0811 02:51:54.516886 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":229,"thread_start_us":113,"threads_started":1}
I20260811 02:51:54.517230 14216 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:51:54.507871 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.138.193:60073 (local address 127.12.138.254:45929)
0811 02:51:54.508010 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0811 02:51:54.508016 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 02:51:54.508563 (+ 547us) server_negotiation.cc:407] Connection header received
0811 02:51:54.508689 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:51:54.508694 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:51:54.508750 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:51:54.508846 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:51:54.509454 (+ 608us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:51:54.510190 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:51:54.511496 (+ 1306us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:51:54.511872 (+ 376us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:51:54.511995 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:51:54.513311 (+ 1316us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:51:54.513317 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:51:54.513320 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:51:54.513351 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:51:54.515193 (+ 1842us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:51:54.515684 (+ 491us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:51:54.515688 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:51:54.515690 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:51:54.515787 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:51:54.516102 (+ 315us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:51:54.516106 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:51:54.516107 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:51:54.516352 (+ 245us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:51:54.516471 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0811 02:51:54.516860 (+ 389us) server_negotiation.cc:299] Negotiation successful
0811 02:51:54.516992 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
WARNING: no policy specified for kudu/127.12.138.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.138.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.138.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786416707705494-12843-0/krb5kdc/kudu_127.12.138.194.keytab.
Entry for principal kudu/127.12.138.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786416707705494-12843-0/krb5kdc/kudu_127.12.138.194.keytab.
I20260811 02:51:54.540307 12843 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDawEwS/build/debug/bin/kudu
/tmp/dist-test-taskDawEwS/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786416707705494-12843-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786416707705494-12843-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786416707705494-12843-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786416707705494-12843-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.138.194:0
--local_ip_for_outbound_sockets=127.12.138.194
--webserver_interface=127.12.138.194
--webserver_port=0
--tserver_master_addrs=127.12.138.254:45929
--keytab_file=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786416707705494-12843-0/krb5kdc/kudu_127.12.138.194.keytab
--principal=kudu/127.12.138.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.138.212:39955
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786416707705494-12843-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786416707705494-12843-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786416707705494-12843-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 02:51:54.656486 14414 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 02:51:54.656823 14414 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 02:51:54.656910 14414 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 02:51:54.661048 14414 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 02:51:54.661185 14414 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 02:51:54.661254 14414 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260811 02:51:54.661316 14414 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.138.194
I20260811 02:51:54.666772 14414 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.138.212:39955
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786416707705494-12843-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786416707705494-12843-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786416707705494-12843-0/krb5kdc/kudu_127.12.138.194.keytab
--principal=kudu/127.12.138.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.138.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786416707705494-12843-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.12.138.194
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.12.138.254:45929
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14414
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.138.194
--log_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786416707705494-12843-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 02:09:49 UTC on ccycloud.quasar-fsnqqc.root.comops.site
build id 4579570
I20260811 02:51:54.667980 14414 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 02:51:54.669106 14414 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 02:51:54.679207 14420 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:51:54.679265 14425 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:51:54.679214 14423 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:51:54.679577 14414 server_base.cc:1034] running on GCE node
I20260811 02:51:54.679983 14414 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 02:51:54.680608 14414 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 02:51:54.681772 14414 hybrid_clock.cc:648] HybridClock initialized: now 1786416714681758 us; error 25 us; skew 500 ppm
Aug 11 02:51:54 dist-test-slave-9gcw krb5kdc[14106](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416714, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.138.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 02:51:54.684839 14414 init.cc:377] Logged in from keytab as kudu/127.12.138.194@KRBTEST.COM (short username kudu)
I20260811 02:51:54.685811 14414 webserver.cc:460] Webserver started at http://127.12.138.194:41195/ using document root <none> and password file <none>
I20260811 02:51:54.686357 14414 fs_manager.cc:359] Metadata directory not provided
I20260811 02:51:54.686416 14414 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 02:51:54.686585 14414 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 02:51:54.688678 14414 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786416707705494-12843-0/minicluster-data/ts-1/data/instance:
uuid: "ad6ef753e62b48c095f130de9398f2b5"
format_stamp: "Formatted at 2026-08-11 02:51:54 on dist-test-slave-9gcw"
server_key: "030aeeed0f32ce1ce3564a7a330b30be"
server_key_iv: "499f436f1d54392f5a71de34ee323c34"
server_key_version: "encryptionkey@0"
I20260811 02:51:54.689298 14414 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786416707705494-12843-0/minicluster-data/ts-1/wal/instance:
uuid: "ad6ef753e62b48c095f130de9398f2b5"
format_stamp: "Formatted at 2026-08-11 02:51:54 on dist-test-slave-9gcw"
server_key: "030aeeed0f32ce1ce3564a7a330b30be"
server_key_iv: "499f436f1d54392f5a71de34ee323c34"
server_key_version: "encryptionkey@0"
I20260811 02:51:54.693302 14414 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260811 02:51:54.695796 14434 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 02:51:54.697029 14414 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260811 02:51:54.697160 14414 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786416707705494-12843-0/minicluster-data/ts-1/data,/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786416707705494-12843-0/minicluster-data/ts-1/wal
uuid: "ad6ef753e62b48c095f130de9398f2b5"
format_stamp: "Formatted at 2026-08-11 02:51:54 on dist-test-slave-9gcw"
server_key: "030aeeed0f32ce1ce3564a7a330b30be"
server_key_iv: "499f436f1d54392f5a71de34ee323c34"
server_key_version: "encryptionkey@0"
I20260811 02:51:54.697312 14414 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786416707705494-12843-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786416707705494-12843-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786416707705494-12843-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 02:51:54.710669 14414 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 02:51:54.713670 14414 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 02:51:54.713908 14414 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 02:51:54.715473 14414 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 02:51:54.715552 14414 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 02:51:54.715615 14414 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 02:51:54.715665 14414 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 11 02:51:54 dist-test-slave-9gcw krb5kdc[14106](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416714, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.138.194@KRBTEST.COM for kudu/127.12.138.254@KRBTEST.COM
I20260811 02:51:54.727190 14414 rpc_server.cc:307] RPC server started. Bound to: 127.12.138.194:42189
I20260811 02:51:54.727244 14589 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.138.194:42189 every 8 connection(s)
I20260811 02:51:54.728155 14414 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786416707705494-12843-0/minicluster-data/ts-1/data/info.pb
I20260811 02:51:54.730425 14216 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:51:54.716363 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.138.194:60949 (local address 127.12.138.254:45929)
0811 02:51:54.716501 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0811 02:51:54.716506 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 02:51:54.717724 (+ 1218us) server_negotiation.cc:407] Connection header received
0811 02:51:54.718939 (+ 1215us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:51:54.718943 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:51:54.719011 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:51:54.719119 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:51:54.720794 (+ 1675us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:51:54.721575 (+ 781us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:51:54.722798 (+ 1223us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:51:54.723144 (+ 346us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:51:54.723274 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:51:54.726320 (+ 3046us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:51:54.726324 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:51:54.726327 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:51:54.726358 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:51:54.728225 (+ 1867us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:51:54.728856 (+ 631us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:51:54.728860 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:51:54.728862 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:51:54.728965 (+ 103us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:51:54.729376 (+ 411us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:51:54.729379 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:51:54.729380 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:51:54.729579 (+ 199us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:51:54.729680 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0811 02:51:54.730181 (+ 501us) server_negotiation.cc:299] Negotiation successful
0811 02:51:54.730303 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260811 02:51:54.731195 14445 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:51:54.716875 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.138.254:45929 (local address 127.12.138.194:60949)
0811 02:51:54.717449 (+ 574us) negotiation.cc:107] Waiting for socket to connect
0811 02:51:54.717498 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0811 02:51:54.718669 (+ 1171us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:51:54.719317 (+ 648us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:51:54.719324 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:51:54.719894 (+ 570us) client_negotiation.cc:189] Negotiated authn=SASL
0811 02:51:54.720618 (+ 724us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:51:54.720656 (+ 38us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:51:54.721724 (+ 1068us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:51:54.721728 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:51:54.722645 (+ 917us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:51:54.722656 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:51:54.723324 (+ 668us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:51:54.723329 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:51:54.723545 (+ 216us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:51:54.724251 (+ 706us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 02:51:54.724274 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0811 02:51:54.726137 (+ 1863us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 02:51:54.728349 (+ 2212us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:51:54.728355 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:51:54.728370 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:51:54.728713 (+ 343us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:51:54.729079 (+ 366us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:51:54.729082 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:51:54.729084 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:51:54.729274 (+ 190us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:51:54.729688 (+ 414us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 02:51:54.729693 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 02:51:54.729917 (+ 224us) client_negotiation.cc:769] Sending connection context
0811 02:51:54.730068 (+ 151us) client_negotiation.cc:240] Negotiation successful
0811 02:51:54.730338 (+ 270us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":367,"thread_start_us":164,"threads_started":1}
I20260811 02:51:54.732465 14590 heartbeater.cc:347] Connected to a master server at 127.12.138.254:45929
I20260811 02:51:54.732635 14590 heartbeater.cc:464] Registering TS with master...
I20260811 02:51:54.733122 14590 heartbeater.cc:511] Master 127.12.138.254:45929 requested a full tablet report, sending...
I20260811 02:51:54.734324 14159 ts_manager.cc:194] Registered new tserver with Master: ad6ef753e62b48c095f130de9398f2b5 (127.12.138.194:42189)
I20260811 02:51:54.735090 14159 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.12.138.194@KRBTEST.COM'} at 127.12.138.194:60949
I20260811 02:51:54.738302 12843 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDawEwS/build/debug/bin/kudu as pid 14414
I20260811 02:51:54.738430 12843 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786416707705494-12843-0/minicluster-data/ts-1/wal/instance
I20260811 02:51:54.738858 12843 external_mini_cluster.cc:1428] Setting key 2920c4c72518e436c97c605019211a94
I20260811 02:51:54.743273 14596 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:51:54.734929 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.138.254:45929 (local address 127.12.138.194:50757)
0811 02:51:54.735223 (+ 294us) negotiation.cc:107] Waiting for socket to connect
0811 02:51:54.735239 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 02:51:54.735386 (+ 147us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:51:54.735658 (+ 272us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:51:54.735661 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:51:54.735896 (+ 235us) client_negotiation.cc:189] Negotiated authn=SASL
0811 02:51:54.736014 (+ 118us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:51:54.736020 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:51:54.737042 (+ 1022us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:51:54.737046 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:51:54.737781 (+ 735us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:51:54.737790 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:51:54.738443 (+ 653us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:51:54.738449 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:51:54.738547 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:51:54.739104 (+ 557us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 02:51:54.739116 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0811 02:51:54.739550 (+ 434us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 02:51:54.741548 (+ 1998us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:51:54.741554 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:51:54.741559 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:51:54.741831 (+ 272us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:51:54.742127 (+ 296us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:51:54.742133 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:51:54.742138 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:51:54.742247 (+ 109us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:51:54.742856 (+ 609us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 02:51:54.742858 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 02:51:54.742936 (+ 78us) client_negotiation.cc:769] Sending connection context
0811 02:51:54.743021 (+ 85us) client_negotiation.cc:240] Negotiation successful
0811 02:51:54.743127 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":210,"mutex_wait_us":7,"thread_start_us":116,"threads_started":1}
I20260811 02:51:54.743481 14216 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:51:54.734994 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.138.194:50757 (local address 127.12.138.254:45929)
0811 02:51:54.735131 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0811 02:51:54.735136 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 02:51:54.735332 (+ 196us) server_negotiation.cc:407] Connection header received
0811 02:51:54.735489 (+ 157us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:51:54.735492 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:51:54.735537 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:51:54.735625 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:51:54.736129 (+ 504us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:51:54.736879 (+ 750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:51:54.737905 (+ 1026us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:51:54.738292 (+ 387us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:51:54.738433 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:51:54.739693 (+ 1260us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:51:54.739697 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:51:54.739699 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:51:54.739725 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:51:54.741439 (+ 1714us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:51:54.741939 (+ 500us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:51:54.741942 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:51:54.741943 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:51:54.742025 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:51:54.742372 (+ 347us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:51:54.742378 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:51:54.742382 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:51:54.742740 (+ 358us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:51:54.742874 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0811 02:51:54.743120 (+ 246us) server_negotiation.cc:299] Negotiation successful
0811 02:51:54.743259 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
WARNING: no policy specified for kudu/127.12.138.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.138.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.138.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786416707705494-12843-0/krb5kdc/kudu_127.12.138.195.keytab.
Entry for principal kudu/127.12.138.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786416707705494-12843-0/krb5kdc/kudu_127.12.138.195.keytab.
I20260811 02:51:54.773440 12843 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDawEwS/build/debug/bin/kudu
/tmp/dist-test-taskDawEwS/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786416707705494-12843-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786416707705494-12843-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786416707705494-12843-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786416707705494-12843-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.138.195:0
--local_ip_for_outbound_sockets=127.12.138.195
--webserver_interface=127.12.138.195
--webserver_port=0
--tserver_master_addrs=127.12.138.254:45929
--keytab_file=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786416707705494-12843-0/krb5kdc/kudu_127.12.138.195.keytab
--principal=kudu/127.12.138.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.138.212:39955
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786416707705494-12843-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786416707705494-12843-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786416707705494-12843-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 02:51:54.894906 14600 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 02:51:54.895258 14600 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 02:51:54.895386 14600 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 02:51:54.900166 14600 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 02:51:54.900264 14600 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 02:51:54.900346 14600 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260811 02:51:54.900486 14600 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.138.195
I20260811 02:51:54.906795 14600 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.138.212:39955
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786416707705494-12843-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786416707705494-12843-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786416707705494-12843-0/krb5kdc/kudu_127.12.138.195.keytab
--principal=kudu/127.12.138.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.138.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786416707705494-12843-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.12.138.195
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.12.138.254:45929
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14600
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.138.195
--log_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786416707705494-12843-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 02:09:49 UTC on ccycloud.quasar-fsnqqc.root.comops.site
build id 4579570
I20260811 02:51:54.907894 14600 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 02:51:54.909083 14600 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 02:51:54.919484 14606 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:51:54.919543 14607 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:51:54.919683 14600 server_base.cc:1034] running on GCE node
W20260811 02:51:54.919554 14610 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:51:54.920388 14600 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 02:51:54.921366 14600 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 02:51:54.922570 14600 hybrid_clock.cc:648] HybridClock initialized: now 1786416714922492 us; error 99 us; skew 500 ppm
Aug 11 02:51:54 dist-test-slave-9gcw krb5kdc[14106](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416714, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.138.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 02:51:54.926537 14600 init.cc:377] Logged in from keytab as kudu/127.12.138.195@KRBTEST.COM (short username kudu)
I20260811 02:51:54.927870 14600 webserver.cc:460] Webserver started at http://127.12.138.195:46733/ using document root <none> and password file <none>
I20260811 02:51:54.928712 14600 fs_manager.cc:359] Metadata directory not provided
I20260811 02:51:54.928861 14600 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 02:51:54.929129 14600 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 02:51:54.931504 14600 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786416707705494-12843-0/minicluster-data/ts-2/data/instance:
uuid: "9f0b981b925841fabaeb66b5c6650494"
format_stamp: "Formatted at 2026-08-11 02:51:54 on dist-test-slave-9gcw"
server_key: "b86b71c5bc365a2d17348eb7f00bbf53"
server_key_iv: "fb7694c60e2a7a34d9f377c65b89b6ea"
server_key_version: "encryptionkey@0"
I20260811 02:51:54.932163 14600 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786416707705494-12843-0/minicluster-data/ts-2/wal/instance:
uuid: "9f0b981b925841fabaeb66b5c6650494"
format_stamp: "Formatted at 2026-08-11 02:51:54 on dist-test-slave-9gcw"
server_key: "b86b71c5bc365a2d17348eb7f00bbf53"
server_key_iv: "fb7694c60e2a7a34d9f377c65b89b6ea"
server_key_version: "encryptionkey@0"
I20260811 02:51:54.936506 14600 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.002s
I20260811 02:51:54.939260 14618 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 02:51:54.940605 14600 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260811 02:51:54.940832 14600 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786416707705494-12843-0/minicluster-data/ts-2/data,/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786416707705494-12843-0/minicluster-data/ts-2/wal
uuid: "9f0b981b925841fabaeb66b5c6650494"
format_stamp: "Formatted at 2026-08-11 02:51:54 on dist-test-slave-9gcw"
server_key: "b86b71c5bc365a2d17348eb7f00bbf53"
server_key_iv: "fb7694c60e2a7a34d9f377c65b89b6ea"
server_key_version: "encryptionkey@0"
I20260811 02:51:54.940958 14600 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786416707705494-12843-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786416707705494-12843-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786416707705494-12843-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 02:51:54.955814 14600 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 02:51:54.959187 14600 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 02:51:54.959403 14600 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 02:51:54.960916 14600 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 02:51:54.960989 14600 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 02:51:54.961062 14600 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 02:51:54.961114 14600 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 11 02:51:54 dist-test-slave-9gcw krb5kdc[14106](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416714, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.138.195@KRBTEST.COM for kudu/127.12.138.254@KRBTEST.COM
I20260811 02:51:54.972879 14600 rpc_server.cc:307] RPC server started. Bound to: 127.12.138.195:41737
I20260811 02:51:54.972993 14755 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.138.195:41737 every 8 connection(s)
I20260811 02:51:54.974102 14600 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786416707705494-12843-0/minicluster-data/ts-2/data/info.pb
I20260811 02:51:54.976353 14216 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:51:54.961899 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.138.195:48527 (local address 127.12.138.254:45929)
0811 02:51:54.962043 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0811 02:51:54.962048 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 02:51:54.963095 (+ 1047us) server_negotiation.cc:407] Connection header received
0811 02:51:54.964402 (+ 1307us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:51:54.964408 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:51:54.964483 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:51:54.964595 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:51:54.966258 (+ 1663us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:51:54.966859 (+ 601us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:51:54.968388 (+ 1529us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:51:54.968704 (+ 316us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:51:54.968835 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:51:54.972066 (+ 3231us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:51:54.972070 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:51:54.972072 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:51:54.972098 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:51:54.973939 (+ 1841us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:51:54.974604 (+ 665us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:51:54.974610 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:51:54.974615 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:51:54.974724 (+ 109us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:51:54.975154 (+ 430us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:51:54.975160 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:51:54.975164 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:51:54.975393 (+ 229us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:51:54.975510 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0811 02:51:54.976045 (+ 535us) server_negotiation.cc:299] Negotiation successful
0811 02:51:54.976176 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260811 02:51:54.977094 14627 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:51:54.962416 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.138.254:45929 (local address 127.12.138.195:48527)
0811 02:51:54.962932 (+ 516us) negotiation.cc:107] Waiting for socket to connect
0811 02:51:54.962978 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0811 02:51:54.964067 (+ 1089us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:51:54.964850 (+ 783us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:51:54.964860 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:51:54.965411 (+ 551us) client_negotiation.cc:189] Negotiated authn=SASL
0811 02:51:54.966089 (+ 678us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:51:54.966108 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:51:54.967031 (+ 923us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:51:54.967039 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:51:54.968249 (+ 1210us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:51:54.968260 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:51:54.968885 (+ 625us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:51:54.968890 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:51:54.969094 (+ 204us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:51:54.969841 (+ 747us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 02:51:54.969863 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0811 02:51:54.971866 (+ 2003us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 02:51:54.974079 (+ 2213us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:51:54.974083 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:51:54.974095 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:51:54.974460 (+ 365us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:51:54.974843 (+ 383us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:51:54.974849 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:51:54.974853 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:51:54.975030 (+ 177us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:51:54.975507 (+ 477us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 02:51:54.975514 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 02:51:54.975764 (+ 250us) client_negotiation.cc:769] Sending connection context
0811 02:51:54.975984 (+ 220us) client_negotiation.cc:240] Negotiation successful
0811 02:51:54.976312 (+ 328us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":308,"thread_start_us":146,"threads_started":1}
I20260811 02:51:54.978286 14757 heartbeater.cc:347] Connected to a master server at 127.12.138.254:45929
I20260811 02:51:54.978451 14757 heartbeater.cc:464] Registering TS with master...
I20260811 02:51:54.978960 14757 heartbeater.cc:511] Master 127.12.138.254:45929 requested a full tablet report, sending...
I20260811 02:51:54.980085 14158 ts_manager.cc:194] Registered new tserver with Master: 9f0b981b925841fabaeb66b5c6650494 (127.12.138.195:41737)
I20260811 02:51:54.980793 14158 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.12.138.195@KRBTEST.COM'} at 127.12.138.195:48527
I20260811 02:51:54.981511 12843 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDawEwS/build/debug/bin/kudu as pid 14600
I20260811 02:51:54.981642 12843 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786416707705494-12843-0/minicluster-data/ts-2/wal/instance
I20260811 02:51:54.981972 12843 external_mini_cluster.cc:1428] Setting key 92415bef961c70073d1ea49dda219579
I20260811 02:51:54.983994 12843 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 02:51:54.988541 14216 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:51:54.980725 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.138.195:45231 (local address 127.12.138.254:45929)
0811 02:51:54.980847 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0811 02:51:54.980853 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 02:51:54.981002 (+ 149us) server_negotiation.cc:407] Connection header received
0811 02:51:54.981190 (+ 188us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:51:54.981192 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:51:54.981238 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:51:54.981329 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:51:54.981882 (+ 553us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:51:54.982599 (+ 717us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:51:54.983551 (+ 952us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:51:54.983860 (+ 309us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:51:54.983978 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:51:54.985108 (+ 1130us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:51:54.985115 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:51:54.985121 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:51:54.985180 (+ 59us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:51:54.986823 (+ 1643us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:51:54.987340 (+ 517us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:51:54.987343 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:51:54.987344 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:51:54.987449 (+ 105us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:51:54.987754 (+ 305us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:51:54.987756 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:51:54.987758 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:51:54.987923 (+ 165us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:51:54.988001 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0811 02:51:54.988335 (+ 334us) server_negotiation.cc:299] Negotiation successful
0811 02:51:54.988430 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260811 02:51:54.988857 14765 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:51:54.980577 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.138.254:45929 (local address 127.12.138.195:45231)
0811 02:51:54.980927 (+ 350us) negotiation.cc:107] Waiting for socket to connect
0811 02:51:54.980949 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0811 02:51:54.981055 (+ 106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:51:54.981367 (+ 312us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:51:54.981372 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:51:54.981604 (+ 232us) client_negotiation.cc:189] Negotiated authn=SASL
0811 02:51:54.981753 (+ 149us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:51:54.981761 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:51:54.982778 (+ 1017us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:51:54.982781 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:51:54.983437 (+ 656us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:51:54.983447 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:51:54.983989 (+ 542us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:51:54.983992 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:51:54.984077 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:51:54.984586 (+ 509us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 02:51:54.984597 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0811 02:51:54.984961 (+ 364us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 02:51:54.986937 (+ 1976us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:51:54.986943 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:51:54.986947 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:51:54.987224 (+ 277us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:51:54.987540 (+ 316us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:51:54.987545 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:51:54.987548 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:51:54.987663 (+ 115us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:51:54.988020 (+ 357us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 02:51:54.988025 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 02:51:54.988143 (+ 118us) client_negotiation.cc:769] Sending connection context
0811 02:51:54.988265 (+ 122us) client_negotiation.cc:240] Negotiation successful
0811 02:51:54.988432 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":224,"thread_start_us":149,"threads_started":1}
Aug 11 02:51:54 dist-test-slave-9gcw krb5kdc[14106](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416714, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 11 02:51:55 dist-test-slave-9gcw krb5kdc[14106](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416714, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.12.138.254@KRBTEST.COM
I20260811 02:51:55.013667 14216 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:51:55.004493 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59746 (local address 127.12.138.254:45929)
0811 02:51:55.004646 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0811 02:51:55.004651 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 02:51:55.004666 (+ 15us) server_negotiation.cc:407] Connection header received
0811 02:51:55.004714 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:51:55.004717 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:51:55.004755 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:51:55.004852 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:51:55.005701 (+ 849us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:51:55.006235 (+ 534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:51:55.007104 (+ 869us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:51:55.007336 (+ 232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:51:55.007444 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:51:55.010068 (+ 2624us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:51:55.010071 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:51:55.010073 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:51:55.010099 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:51:55.011588 (+ 1489us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:51:55.012110 (+ 522us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:51:55.012115 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:51:55.012117 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:51:55.012215 (+ 98us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:51:55.012547 (+ 332us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:51:55.012551 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:51:55.012553 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:51:55.012839 (+ 286us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:51:55.013003 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0811 02:51:55.013328 (+ 325us) server_negotiation.cc:299] Negotiation successful
0811 02:51:55.013471 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260811 02:51:55.016903 14158 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:59746:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 02:51:55.020429 14158 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 02:51:55.036046 14776 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:51:55.029404 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.138.193:43377 (local address 127.0.0.1:54586)
0811 02:51:55.030349 (+ 945us) negotiation.cc:107] Waiting for socket to connect
0811 02:51:55.030362 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 02:51:55.030458 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:51:55.031210 (+ 752us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:51:55.031213 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:51:55.031243 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 02:51:55.031448 (+ 205us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:51:55.031455 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:51:55.032702 (+ 1247us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:51:55.032706 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:51:55.034367 (+ 1661us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:51:55.034378 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:51:55.035418 (+ 1040us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:51:55.035422 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:51:55.035532 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:51:55.035571 (+ 39us) client_negotiation.cc:769] Sending connection context
0811 02:51:55.035724 (+ 153us) client_negotiation.cc:240] Negotiation successful
0811 02:51:55.035859 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":878,"mutex_wait_us":98,"spinlock_wait_cycles":7296,"thread_start_us":80,"threads_started":1}
I20260811 02:51:55.036021 14781 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:51:55.030194 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54586 (local address 127.12.138.193:43377)
0811 02:51:55.030685 (+ 491us) server_negotiation.cc:206] Beginning negotiation
0811 02:51:55.030693 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 02:51:55.030719 (+ 26us) server_negotiation.cc:407] Connection header received
0811 02:51:55.030798 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:51:55.030804 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:51:55.031036 (+ 232us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:51:55.031607 (+ 571us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 02:51:55.031744 (+ 137us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:51:55.032580 (+ 836us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:51:55.034542 (+ 1962us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:51:55.035275 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:51:55.035434 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:51:55.035587 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0811 02:51:55.035756 (+ 169us) server_negotiation.cc:299] Negotiation successful
0811 02:51:55.035859 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":386,"thread_start_us":66,"threads_started":1}
I20260811 02:51:55.036119 14777 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:51:55.029319 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.138.195:41737 (local address 127.0.0.1:36380)
0811 02:51:55.030188 (+ 869us) negotiation.cc:107] Waiting for socket to connect
0811 02:51:55.030243 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0811 02:51:55.030374 (+ 131us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:51:55.031439 (+ 1065us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:51:55.031441 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:51:55.031459 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 02:51:55.031552 (+ 93us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:51:55.031558 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:51:55.033011 (+ 1453us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:51:55.033015 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:51:55.034385 (+ 1370us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:51:55.034396 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:51:55.035578 (+ 1182us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:51:55.035582 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:51:55.035731 (+ 149us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:51:55.035782 (+ 51us) client_negotiation.cc:769] Sending connection context
0811 02:51:55.035849 (+ 67us) client_negotiation.cc:240] Negotiation successful
0811 02:51:55.035922 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":681,"mutex_wait_us":218,"thread_start_us":86,"threads_started":1}
I20260811 02:51:55.036635 14779 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:51:55.029656 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.138.194:42189 (local address 127.0.0.1:47394)
0811 02:51:55.030590 (+ 934us) negotiation.cc:107] Waiting for socket to connect
0811 02:51:55.030605 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 02:51:55.030692 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:51:55.031200 (+ 508us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:51:55.031206 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:51:55.031239 (+ 33us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 02:51:55.031428 (+ 189us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:51:55.031436 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:51:55.032965 (+ 1529us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:51:55.032968 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:51:55.034379 (+ 1411us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:51:55.034387 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:51:55.035595 (+ 1208us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:51:55.035598 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:51:55.035726 (+ 128us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:51:55.036315 (+ 589us) client_negotiation.cc:769] Sending connection context
0811 02:51:55.036408 (+ 93us) client_negotiation.cc:240] Negotiation successful
0811 02:51:55.036485 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":865,"mutex_wait_us":685,"spinlock_wait_cycles":2816,"thread_start_us":91,"threads_started":1}
I20260811 02:51:55.037448 14780 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:51:55.030034 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47394 (local address 127.12.138.194:42189)
0811 02:51:55.030684 (+ 650us) server_negotiation.cc:206] Beginning negotiation
0811 02:51:55.030697 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0811 02:51:55.030719 (+ 22us) server_negotiation.cc:407] Connection header received
0811 02:51:55.030799 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:51:55.030804 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:51:55.031031 (+ 227us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:51:55.031210 (+ 179us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 02:51:55.031571 (+ 361us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:51:55.032859 (+ 1288us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:51:55.034505 (+ 1646us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:51:55.035438 (+ 933us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:51:55.035853 (+ 415us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:51:55.035988 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0811 02:51:55.037230 (+ 1242us) server_negotiation.cc:299] Negotiation successful
0811 02:51:55.037339 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":539,"thread_start_us":115,"threads_started":1}
I20260811 02:51:55.038395 14778 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:51:55.029457 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36380 (local address 127.12.138.195:41737)
0811 02:51:55.030956 (+ 1499us) server_negotiation.cc:206] Beginning negotiation
0811 02:51:55.030969 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0811 02:51:55.030992 (+ 23us) server_negotiation.cc:407] Connection header received
0811 02:51:55.031067 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:51:55.031071 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:51:55.031279 (+ 208us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:51:55.031455 (+ 176us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 02:51:55.031662 (+ 207us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:51:55.032845 (+ 1183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:51:55.034541 (+ 1696us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:51:55.035425 (+ 884us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:51:55.036954 (+ 1529us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:51:55.037101 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0811 02:51:55.038133 (+ 1032us) server_negotiation.cc:299] Negotiation successful
0811 02:51:55.038240 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1389,"thread_start_us":132,"threads_started":1}
I20260811 02:51:55.039268 14318 tablet_service.cc:1467] Processing CreateTablet for tablet 77b2803d67c84354b948378051d281c3 (DEFAULT_TABLE table=test-table [id=c73727cccb314ab9b712358bcded99d0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 02:51:55.039949 14500 tablet_service.cc:1467] Processing CreateTablet for tablet 77b2803d67c84354b948378051d281c3 (DEFAULT_TABLE table=test-table [id=c73727cccb314ab9b712358bcded99d0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 02:51:55.040524 14318 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 77b2803d67c84354b948378051d281c3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 02:51:55.040966 14500 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 77b2803d67c84354b948378051d281c3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 02:51:55.040964 14679 tablet_service.cc:1467] Processing CreateTablet for tablet 77b2803d67c84354b948378051d281c3 (DEFAULT_TABLE table=test-table [id=c73727cccb314ab9b712358bcded99d0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 02:51:55.042043 14679 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 77b2803d67c84354b948378051d281c3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 02:51:55.048385 14783 tablet_bootstrap.cc:492] T 77b2803d67c84354b948378051d281c3 P ad6ef753e62b48c095f130de9398f2b5: Bootstrap starting.
I20260811 02:51:55.049824 14784 tablet_bootstrap.cc:492] T 77b2803d67c84354b948378051d281c3 P c9b7177680454e789d9e860ba91be083: Bootstrap starting.
I20260811 02:51:55.050024 14785 tablet_bootstrap.cc:492] T 77b2803d67c84354b948378051d281c3 P 9f0b981b925841fabaeb66b5c6650494: Bootstrap starting.
I20260811 02:51:55.051636 14783 tablet_bootstrap.cc:654] T 77b2803d67c84354b948378051d281c3 P ad6ef753e62b48c095f130de9398f2b5: Neither blocks nor log segments found. Creating new log.
I20260811 02:51:55.051683 14784 tablet_bootstrap.cc:654] T 77b2803d67c84354b948378051d281c3 P c9b7177680454e789d9e860ba91be083: Neither blocks nor log segments found. Creating new log.
I20260811 02:51:55.052014 14785 tablet_bootstrap.cc:654] T 77b2803d67c84354b948378051d281c3 P 9f0b981b925841fabaeb66b5c6650494: Neither blocks nor log segments found. Creating new log.
I20260811 02:51:55.052472 14784 log.cc:824] T 77b2803d67c84354b948378051d281c3 P c9b7177680454e789d9e860ba91be083: Log is configured to *not* fsync() on all Append() calls
I20260811 02:51:55.052529 14783 log.cc:824] T 77b2803d67c84354b948378051d281c3 P ad6ef753e62b48c095f130de9398f2b5: Log is configured to *not* fsync() on all Append() calls
I20260811 02:51:55.052760 14785 log.cc:824] T 77b2803d67c84354b948378051d281c3 P 9f0b981b925841fabaeb66b5c6650494: Log is configured to *not* fsync() on all Append() calls
I20260811 02:51:55.054646 14785 tablet_bootstrap.cc:492] T 77b2803d67c84354b948378051d281c3 P 9f0b981b925841fabaeb66b5c6650494: No bootstrap required, opened a new log
I20260811 02:51:55.054689 14783 tablet_bootstrap.cc:492] T 77b2803d67c84354b948378051d281c3 P ad6ef753e62b48c095f130de9398f2b5: No bootstrap required, opened a new log
I20260811 02:51:55.054685 14784 tablet_bootstrap.cc:492] T 77b2803d67c84354b948378051d281c3 P c9b7177680454e789d9e860ba91be083: No bootstrap required, opened a new log
I20260811 02:51:55.054869 14783 ts_tablet_manager.cc:1397] T 77b2803d67c84354b948378051d281c3 P ad6ef753e62b48c095f130de9398f2b5: Time spent bootstrapping tablet: real 0.007s user 0.004s sys 0.000s
I20260811 02:51:55.054869 14784 ts_tablet_manager.cc:1397] T 77b2803d67c84354b948378051d281c3 P c9b7177680454e789d9e860ba91be083: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260811 02:51:55.054874 14785 ts_tablet_manager.cc:1397] T 77b2803d67c84354b948378051d281c3 P 9f0b981b925841fabaeb66b5c6650494: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260811 02:51:55.058441 14785 raft_consensus.cc:348] T 77b2803d67c84354b948378051d281c3 P 9f0b981b925841fabaeb66b5c6650494 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f0b981b925841fabaeb66b5c6650494" member_type: VOTER last_known_addr { host: "127.12.138.195" port: 41737 } } peers { permanent_uuid: "c9b7177680454e789d9e860ba91be083" member_type: VOTER last_known_addr { host: "127.12.138.193" port: 43377 } } peers { permanent_uuid: "ad6ef753e62b48c095f130de9398f2b5" member_type: VOTER last_known_addr { host: "127.12.138.194" port: 42189 } }
I20260811 02:51:55.058704 14785 raft_consensus.cc:374] T 77b2803d67c84354b948378051d281c3 P 9f0b981b925841fabaeb66b5c6650494 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 02:51:55.058796 14785 raft_consensus.cc:729] T 77b2803d67c84354b948378051d281c3 P 9f0b981b925841fabaeb66b5c6650494 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9f0b981b925841fabaeb66b5c6650494, State: Initialized, Role: FOLLOWER
I20260811 02:51:55.058969 14783 raft_consensus.cc:348] T 77b2803d67c84354b948378051d281c3 P ad6ef753e62b48c095f130de9398f2b5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f0b981b925841fabaeb66b5c6650494" member_type: VOTER last_known_addr { host: "127.12.138.195" port: 41737 } } peers { permanent_uuid: "c9b7177680454e789d9e860ba91be083" member_type: VOTER last_known_addr { host: "127.12.138.193" port: 43377 } } peers { permanent_uuid: "ad6ef753e62b48c095f130de9398f2b5" member_type: VOTER last_known_addr { host: "127.12.138.194" port: 42189 } }
I20260811 02:51:55.059226 14783 raft_consensus.cc:374] T 77b2803d67c84354b948378051d281c3 P ad6ef753e62b48c095f130de9398f2b5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 02:51:55.059285 14783 raft_consensus.cc:729] T 77b2803d67c84354b948378051d281c3 P ad6ef753e62b48c095f130de9398f2b5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ad6ef753e62b48c095f130de9398f2b5, State: Initialized, Role: FOLLOWER
I20260811 02:51:55.059378 14785 consensus_queue.cc:260] T 77b2803d67c84354b948378051d281c3 P 9f0b981b925841fabaeb66b5c6650494 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f0b981b925841fabaeb66b5c6650494" member_type: VOTER last_known_addr { host: "127.12.138.195" port: 41737 } } peers { permanent_uuid: "c9b7177680454e789d9e860ba91be083" member_type: VOTER last_known_addr { host: "127.12.138.193" port: 43377 } } peers { permanent_uuid: "ad6ef753e62b48c095f130de9398f2b5" member_type: VOTER last_known_addr { host: "127.12.138.194" port: 42189 } }
I20260811 02:51:55.059418 14784 raft_consensus.cc:348] T 77b2803d67c84354b948378051d281c3 P c9b7177680454e789d9e860ba91be083 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f0b981b925841fabaeb66b5c6650494" member_type: VOTER last_known_addr { host: "127.12.138.195" port: 41737 } } peers { permanent_uuid: "c9b7177680454e789d9e860ba91be083" member_type: VOTER last_known_addr { host: "127.12.138.193" port: 43377 } } peers { permanent_uuid: "ad6ef753e62b48c095f130de9398f2b5" member_type: VOTER last_known_addr { host: "127.12.138.194" port: 42189 } }
I20260811 02:51:55.059659 14784 raft_consensus.cc:374] T 77b2803d67c84354b948378051d281c3 P c9b7177680454e789d9e860ba91be083 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 02:51:55.059715 14784 raft_consensus.cc:729] T 77b2803d67c84354b948378051d281c3 P c9b7177680454e789d9e860ba91be083 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c9b7177680454e789d9e860ba91be083, State: Initialized, Role: FOLLOWER
I20260811 02:51:55.059754 14783 consensus_queue.cc:260] T 77b2803d67c84354b948378051d281c3 P ad6ef753e62b48c095f130de9398f2b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f0b981b925841fabaeb66b5c6650494" member_type: VOTER last_known_addr { host: "127.12.138.195" port: 41737 } } peers { permanent_uuid: "c9b7177680454e789d9e860ba91be083" member_type: VOTER last_known_addr { host: "127.12.138.193" port: 43377 } } peers { permanent_uuid: "ad6ef753e62b48c095f130de9398f2b5" member_type: VOTER last_known_addr { host: "127.12.138.194" port: 42189 } }
I20260811 02:51:55.060176 14757 heartbeater.cc:503] Master 127.12.138.254:45929 was elected leader, sending a full tablet report...
I20260811 02:51:55.060165 14784 consensus_queue.cc:260] T 77b2803d67c84354b948378051d281c3 P c9b7177680454e789d9e860ba91be083 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f0b981b925841fabaeb66b5c6650494" member_type: VOTER last_known_addr { host: "127.12.138.195" port: 41737 } } peers { permanent_uuid: "c9b7177680454e789d9e860ba91be083" member_type: VOTER last_known_addr { host: "127.12.138.193" port: 43377 } } peers { permanent_uuid: "ad6ef753e62b48c095f130de9398f2b5" member_type: VOTER last_known_addr { host: "127.12.138.194" port: 42189 } }
I20260811 02:51:55.060468 14785 ts_tablet_manager.cc:1428] T 77b2803d67c84354b948378051d281c3 P 9f0b981b925841fabaeb66b5c6650494: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260811 02:51:55.060735 14783 ts_tablet_manager.cc:1428] T 77b2803d67c84354b948378051d281c3 P ad6ef753e62b48c095f130de9398f2b5: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260811 02:51:55.060988 14784 ts_tablet_manager.cc:1428] T 77b2803d67c84354b948378051d281c3 P c9b7177680454e789d9e860ba91be083: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260811 02:51:55.061092 14590 heartbeater.cc:503] Master 127.12.138.254:45929 was elected leader, sending a full tablet report...
I20260811 02:51:55.061322 14399 heartbeater.cc:503] Master 127.12.138.254:45929 was elected leader, sending a full tablet report...
I20260811 02:51:55.079079 14792 raft_consensus.cc:482] T 77b2803d67c84354b948378051d281c3 P 9f0b981b925841fabaeb66b5c6650494 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 02:51:55.079277 14792 raft_consensus.cc:504] T 77b2803d67c84354b948378051d281c3 P 9f0b981b925841fabaeb66b5c6650494 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f0b981b925841fabaeb66b5c6650494" member_type: VOTER last_known_addr { host: "127.12.138.195" port: 41737 } } peers { permanent_uuid: "c9b7177680454e789d9e860ba91be083" member_type: VOTER last_known_addr { host: "127.12.138.193" port: 43377 } } peers { permanent_uuid: "ad6ef753e62b48c095f130de9398f2b5" member_type: VOTER last_known_addr { host: "127.12.138.194" port: 42189 } }
I20260811 02:51:55.080494 14792 leader_election.cc:288] T 77b2803d67c84354b948378051d281c3 P 9f0b981b925841fabaeb66b5c6650494 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c9b7177680454e789d9e860ba91be083 (127.12.138.193:43377), ad6ef753e62b48c095f130de9398f2b5 (127.12.138.194:42189)
I20260811 02:51:55.085006 14781 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:51:55.080795 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.138.195:37057 (local address 127.12.138.193:43377)
0811 02:51:55.080972 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0811 02:51:55.080979 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 02:51:55.080997 (+ 18us) server_negotiation.cc:407] Connection header received
0811 02:51:55.081190 (+ 193us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:51:55.081193 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:51:55.081232 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:51:55.081316 (+ 84us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 02:51:55.081685 (+ 369us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:51:55.082382 (+ 697us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:51:55.083845 (+ 1463us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:51:55.084456 (+ 611us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:51:55.084581 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:51:55.084653 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0811 02:51:55.084841 (+ 188us) server_negotiation.cc:299] Negotiation successful
0811 02:51:55.084904 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":99}
I20260811 02:51:55.085016 14627 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:51:55.080756 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.138.193:43377 (local address 127.12.138.195:37057)
0811 02:51:55.080930 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0811 02:51:55.080946 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 02:51:55.081042 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:51:55.081340 (+ 298us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:51:55.081343 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:51:55.081387 (+ 44us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 02:51:55.081544 (+ 157us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:51:55.081552 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:51:55.082538 (+ 986us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:51:55.082541 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:51:55.083704 (+ 1163us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:51:55.083714 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:51:55.084574 (+ 860us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:51:55.084576 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:51:55.084694 (+ 118us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:51:55.084721 (+ 27us) client_negotiation.cc:769] Sending connection context
0811 02:51:55.084803 (+ 82us) client_negotiation.cc:240] Negotiation successful
0811 02:51:55.084873 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":89,"mutex_wait_us":25,"spinlock_wait_cycles":7168}
I20260811 02:51:55.085742 14345 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "77b2803d67c84354b948378051d281c3" candidate_uuid: "9f0b981b925841fabaeb66b5c6650494" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c9b7177680454e789d9e860ba91be083" is_pre_election: true
I20260811 02:51:55.085866 14780 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:51:55.080900 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.138.195:38731 (local address 127.12.138.194:42189)
0811 02:51:55.081027 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0811 02:51:55.081032 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 02:51:55.081328 (+ 296us) server_negotiation.cc:407] Connection header received
0811 02:51:55.081518 (+ 190us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:51:55.081521 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:51:55.081560 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:51:55.081646 (+ 86us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 02:51:55.082538 (+ 892us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:51:55.083298 (+ 760us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:51:55.084695 (+ 1397us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:51:55.085322 (+ 627us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:51:55.085464 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:51:55.085514 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0811 02:51:55.085693 (+ 179us) server_negotiation.cc:299] Negotiation successful
0811 02:51:55.085760 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260811 02:51:55.085884 14796 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:51:55.080779 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.138.194:42189 (local address 127.12.138.195:38731)
0811 02:51:55.081228 (+ 449us) negotiation.cc:107] Waiting for socket to connect
0811 02:51:55.081241 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 02:51:55.081391 (+ 150us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:51:55.081657 (+ 266us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:51:55.081660 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:51:55.081675 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 02:51:55.082399 (+ 724us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:51:55.082409 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:51:55.083411 (+ 1002us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:51:55.083413 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:51:55.084572 (+ 1159us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:51:55.084582 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:51:55.085451 (+ 869us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:51:55.085455 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:51:55.085556 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:51:55.085576 (+ 20us) client_negotiation.cc:769] Sending connection context
0811 02:51:55.085677 (+ 101us) client_negotiation.cc:240] Negotiation successful
0811 02:51:55.085744 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":343,"mutex_wait_us":40,"spinlock_wait_cycles":35456,"thread_start_us":146,"threads_started":1}
I20260811 02:51:55.086082 14345 raft_consensus.cc:2454] T 77b2803d67c84354b948378051d281c3 P c9b7177680454e789d9e860ba91be083 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9f0b981b925841fabaeb66b5c6650494 in term 0.
I20260811 02:51:55.086534 14528 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "77b2803d67c84354b948378051d281c3" candidate_uuid: "9f0b981b925841fabaeb66b5c6650494" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ad6ef753e62b48c095f130de9398f2b5" is_pre_election: true
I20260811 02:51:55.086565 14622 leader_election.cc:302] T 77b2803d67c84354b948378051d281c3 P 9f0b981b925841fabaeb66b5c6650494 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9f0b981b925841fabaeb66b5c6650494, c9b7177680454e789d9e860ba91be083; no voters:
I20260811 02:51:55.086858 14528 raft_consensus.cc:2454] T 77b2803d67c84354b948378051d281c3 P ad6ef753e62b48c095f130de9398f2b5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9f0b981b925841fabaeb66b5c6650494 in term 0.
I20260811 02:51:55.086841 14792 raft_consensus.cc:2781] T 77b2803d67c84354b948378051d281c3 P 9f0b981b925841fabaeb66b5c6650494 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 02:51:55.086975 14792 raft_consensus.cc:482] T 77b2803d67c84354b948378051d281c3 P 9f0b981b925841fabaeb66b5c6650494 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 02:51:55.087029 14792 raft_consensus.cc:3037] T 77b2803d67c84354b948378051d281c3 P 9f0b981b925841fabaeb66b5c6650494 [term 0 FOLLOWER]: Advancing to term 1
I20260811 02:51:55.088341 14792 raft_consensus.cc:504] T 77b2803d67c84354b948378051d281c3 P 9f0b981b925841fabaeb66b5c6650494 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f0b981b925841fabaeb66b5c6650494" member_type: VOTER last_known_addr { host: "127.12.138.195" port: 41737 } } peers { permanent_uuid: "c9b7177680454e789d9e860ba91be083" member_type: VOTER last_known_addr { host: "127.12.138.193" port: 43377 } } peers { permanent_uuid: "ad6ef753e62b48c095f130de9398f2b5" member_type: VOTER last_known_addr { host: "127.12.138.194" port: 42189 } }
I20260811 02:51:55.088689 14792 leader_election.cc:288] T 77b2803d67c84354b948378051d281c3 P 9f0b981b925841fabaeb66b5c6650494 [CANDIDATE]: Term 1 election: Requested vote from peers c9b7177680454e789d9e860ba91be083 (127.12.138.193:43377), ad6ef753e62b48c095f130de9398f2b5 (127.12.138.194:42189)
I20260811 02:51:55.089012 14345 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "77b2803d67c84354b948378051d281c3" candidate_uuid: "9f0b981b925841fabaeb66b5c6650494" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c9b7177680454e789d9e860ba91be083"
I20260811 02:51:55.089047 14528 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "77b2803d67c84354b948378051d281c3" candidate_uuid: "9f0b981b925841fabaeb66b5c6650494" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ad6ef753e62b48c095f130de9398f2b5"
I20260811 02:51:55.089192 14528 raft_consensus.cc:3037] T 77b2803d67c84354b948378051d281c3 P ad6ef753e62b48c095f130de9398f2b5 [term 0 FOLLOWER]: Advancing to term 1
I20260811 02:51:55.089200 14345 raft_consensus.cc:3037] T 77b2803d67c84354b948378051d281c3 P c9b7177680454e789d9e860ba91be083 [term 0 FOLLOWER]: Advancing to term 1
I20260811 02:51:55.090360 14345 raft_consensus.cc:2454] T 77b2803d67c84354b948378051d281c3 P c9b7177680454e789d9e860ba91be083 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9f0b981b925841fabaeb66b5c6650494 in term 1.
I20260811 02:51:55.090758 14528 raft_consensus.cc:2454] T 77b2803d67c84354b948378051d281c3 P ad6ef753e62b48c095f130de9398f2b5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9f0b981b925841fabaeb66b5c6650494 in term 1.
I20260811 02:51:55.090749 14622 leader_election.cc:302] T 77b2803d67c84354b948378051d281c3 P 9f0b981b925841fabaeb66b5c6650494 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9f0b981b925841fabaeb66b5c6650494, c9b7177680454e789d9e860ba91be083; no voters:
I20260811 02:51:55.090938 14792 raft_consensus.cc:2781] T 77b2803d67c84354b948378051d281c3 P 9f0b981b925841fabaeb66b5c6650494 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 02:51:55.091212 14792 raft_consensus.cc:686] T 77b2803d67c84354b948378051d281c3 P 9f0b981b925841fabaeb66b5c6650494 [term 1 LEADER]: Becoming Leader. State: Replica: 9f0b981b925841fabaeb66b5c6650494, State: Running, Role: LEADER
I20260811 02:51:55.091549 14792 consensus_queue.cc:237] T 77b2803d67c84354b948378051d281c3 P 9f0b981b925841fabaeb66b5c6650494 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f0b981b925841fabaeb66b5c6650494" member_type: VOTER last_known_addr { host: "127.12.138.195" port: 41737 } } peers { permanent_uuid: "c9b7177680454e789d9e860ba91be083" member_type: VOTER last_known_addr { host: "127.12.138.193" port: 43377 } } peers { permanent_uuid: "ad6ef753e62b48c095f130de9398f2b5" member_type: VOTER last_known_addr { host: "127.12.138.194" port: 42189 } }
I20260811 02:51:55.095074 14159 catalog_manager.cc:5495] T 77b2803d67c84354b948378051d281c3 P 9f0b981b925841fabaeb66b5c6650494 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9f0b981b925841fabaeb66b5c6650494 (127.12.138.195). New cstate: current_term: 1 leader_uuid: "9f0b981b925841fabaeb66b5c6650494" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f0b981b925841fabaeb66b5c6650494" member_type: VOTER last_known_addr { host: "127.12.138.195" port: 41737 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c9b7177680454e789d9e860ba91be083" member_type: VOTER last_known_addr { host: "127.12.138.193" port: 43377 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ad6ef753e62b48c095f130de9398f2b5" member_type: VOTER last_known_addr { host: "127.12.138.194" port: 42189 } health_report { overall_health: UNKNOWN } } }
I20260811 02:51:55.124979 14778 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:51:55.121344 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36394 (local address 127.12.138.195:41737)
0811 02:51:55.121489 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0811 02:51:55.121493 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 02:51:55.121508 (+ 15us) server_negotiation.cc:407] Connection header received
0811 02:51:55.121651 (+ 143us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:51:55.121654 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:51:55.121711 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:51:55.121814 (+ 103us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 02:51:55.122107 (+ 293us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:51:55.122638 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:51:55.123503 (+ 865us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:51:55.123745 (+ 242us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:51:55.123832 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:51:55.124164 (+ 332us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 02:51:55.124598 (+ 434us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 02:51:55.124709 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0811 02:51:55.124785 (+ 76us) server_negotiation.cc:299] Negotiation successful
0811 02:51:55.124838 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260811 02:51:55.131192 14345 raft_consensus.cc:1261] T 77b2803d67c84354b948378051d281c3 P c9b7177680454e789d9e860ba91be083 [term 1 FOLLOWER]: Refusing update from remote peer 9f0b981b925841fabaeb66b5c6650494: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 02:51:55.131192 14528 raft_consensus.cc:1261] T 77b2803d67c84354b948378051d281c3 P ad6ef753e62b48c095f130de9398f2b5 [term 1 FOLLOWER]: Refusing update from remote peer 9f0b981b925841fabaeb66b5c6650494: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 02:51:55.131992 14792 consensus_queue.cc:1035] T 77b2803d67c84354b948378051d281c3 P 9f0b981b925841fabaeb66b5c6650494 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad6ef753e62b48c095f130de9398f2b5" member_type: VOTER last_known_addr { host: "127.12.138.194" port: 42189 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 02:51:55.132228 14797 consensus_queue.cc:1035] T 77b2803d67c84354b948378051d281c3 P 9f0b981b925841fabaeb66b5c6650494 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c9b7177680454e789d9e860ba91be083" member_type: VOTER last_known_addr { host: "127.12.138.193" port: 43377 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 02:51:55.141381 14801 mvcc.cc:205] Tried to move back new op lower bound from 7317162865168928768 to 7317162865017307136. Current Snapshot: MvccSnapshot[applied={T|T < 7317162865168928768}]
I20260811 02:51:55.152724 14158 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:59746:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 02:51:55.152895 14158 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:59746:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 02:51:55.155594 14158 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 7514043a26cd4e49b389e3b8af39c2ed: Sending DeleteTablet for 3 replicas of tablet 77b2803d67c84354b948378051d281c3
I20260811 02:51:55.156426 14679 tablet_service.cc:1514] Processing DeleteTablet for tablet 77b2803d67c84354b948378051d281c3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 02:51:55 UTC) from {username='kudu', principal='kudu/127.12.138.254@KRBTEST.COM'} at 127.0.0.1:36380
I20260811 02:51:55.156435 14318 tablet_service.cc:1514] Processing DeleteTablet for tablet 77b2803d67c84354b948378051d281c3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 02:51:55 UTC) from {username='kudu', principal='kudu/127.12.138.254@KRBTEST.COM'} at 127.0.0.1:54586
I20260811 02:51:55.156991 14814 tablet_replica.cc:330] T 77b2803d67c84354b948378051d281c3 P c9b7177680454e789d9e860ba91be083: stopping tablet replica
I20260811 02:51:55.157236 14815 tablet_replica.cc:330] T 77b2803d67c84354b948378051d281c3 P 9f0b981b925841fabaeb66b5c6650494: stopping tablet replica
I20260811 02:51:55.157228 14158 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:59746:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 02:51:55.157327 14500 tablet_service.cc:1514] Processing DeleteTablet for tablet 77b2803d67c84354b948378051d281c3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 02:51:55 UTC) from {username='kudu', principal='kudu/127.12.138.254@KRBTEST.COM'} at 127.0.0.1:47394
W20260811 02:51:55.157650 14158 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 02:51:55.157930 14814 raft_consensus.cc:2229] T 77b2803d67c84354b948378051d281c3 P c9b7177680454e789d9e860ba91be083 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 02:51:55.158044 14816 tablet_replica.cc:330] T 77b2803d67c84354b948378051d281c3 P ad6ef753e62b48c095f130de9398f2b5: stopping tablet replica
I20260811 02:51:55.158077 14815 raft_consensus.cc:2229] T 77b2803d67c84354b948378051d281c3 P 9f0b981b925841fabaeb66b5c6650494 [term 1 LEADER]: Raft consensus shutting down.
I20260811 02:51:55.158322 14816 raft_consensus.cc:2229] T 77b2803d67c84354b948378051d281c3 P ad6ef753e62b48c095f130de9398f2b5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 02:51:55.158595 14816 raft_consensus.cc:2258] T 77b2803d67c84354b948378051d281c3 P ad6ef753e62b48c095f130de9398f2b5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 02:51:55.158833 14814 raft_consensus.cc:2258] T 77b2803d67c84354b948378051d281c3 P c9b7177680454e789d9e860ba91be083 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 02:51:55.160120 14816 ts_tablet_manager.cc:1905] T 77b2803d67c84354b948378051d281c3 P ad6ef753e62b48c095f130de9398f2b5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 02:51:55.160761 14815 raft_consensus.cc:2258] T 77b2803d67c84354b948378051d281c3 P 9f0b981b925841fabaeb66b5c6650494 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 02:51:55.163126 14816 ts_tablet_manager.cc:1918] T 77b2803d67c84354b948378051d281c3 P ad6ef753e62b48c095f130de9398f2b5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 02:51:55.163233 14816 log.cc:1196] T 77b2803d67c84354b948378051d281c3 P ad6ef753e62b48c095f130de9398f2b5: Deleting WAL directory at /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786416707705494-12843-0/minicluster-data/ts-1/wal/wals/77b2803d67c84354b948378051d281c3
I20260811 02:51:55.163164 14814 ts_tablet_manager.cc:1905] T 77b2803d67c84354b948378051d281c3 P c9b7177680454e789d9e860ba91be083: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 02:51:55.163568 14816 ts_tablet_manager.cc:1939] T 77b2803d67c84354b948378051d281c3 P ad6ef753e62b48c095f130de9398f2b5: Deleting consensus metadata
I20260811 02:51:55.164623 14318 tablet_service.cc:1467] Processing CreateTablet for tablet 3466ff551c5a420eaae8ed66b21fc0d0 (DEFAULT_TABLE table=test-table [id=46e8968786b244a7a00e91c58f61be28]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 02:51:55.164770 14138 catalog_manager.cc:4841] TS ad6ef753e62b48c095f130de9398f2b5 (127.12.138.194:42189): tablet 77b2803d67c84354b948378051d281c3 (table test-table [id=c73727cccb314ab9b712358bcded99d0]) successfully deleted
I20260811 02:51:55.164923 14318 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3466ff551c5a420eaae8ed66b21fc0d0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 02:51:55.166138 14815 ts_tablet_manager.cc:1905] T 77b2803d67c84354b948378051d281c3 P 9f0b981b925841fabaeb66b5c6650494: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 02:51:55.166435 14814 ts_tablet_manager.cc:1918] T 77b2803d67c84354b948378051d281c3 P c9b7177680454e789d9e860ba91be083: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 02:51:55.166581 14814 log.cc:1196] T 77b2803d67c84354b948378051d281c3 P c9b7177680454e789d9e860ba91be083: Deleting WAL directory at /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786416707705494-12843-0/minicluster-data/ts-0/wal/wals/77b2803d67c84354b948378051d281c3
I20260811 02:51:55.166953 14814 ts_tablet_manager.cc:1939] T 77b2803d67c84354b948378051d281c3 P c9b7177680454e789d9e860ba91be083: Deleting consensus metadata
I20260811 02:51:55.167541 14784 tablet_bootstrap.cc:492] T 3466ff551c5a420eaae8ed66b21fc0d0 P c9b7177680454e789d9e860ba91be083: Bootstrap starting.
I20260811 02:51:55.167572 14679 tablet_service.cc:1467] Processing CreateTablet for tablet 3466ff551c5a420eaae8ed66b21fc0d0 (DEFAULT_TABLE table=test-table [id=46e8968786b244a7a00e91c58f61be28]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 02:51:55.167815 14679 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3466ff551c5a420eaae8ed66b21fc0d0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 02:51:55.168550 14784 tablet_bootstrap.cc:654] T 3466ff551c5a420eaae8ed66b21fc0d0 P c9b7177680454e789d9e860ba91be083: Neither blocks nor log segments found. Creating new log.
I20260811 02:51:55.169001 14815 ts_tablet_manager.cc:1918] T 77b2803d67c84354b948378051d281c3 P 9f0b981b925841fabaeb66b5c6650494: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 02:51:55.169088 14815 log.cc:1196] T 77b2803d67c84354b948378051d281c3 P 9f0b981b925841fabaeb66b5c6650494: Deleting WAL directory at /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786416707705494-12843-0/minicluster-data/ts-2/wal/wals/77b2803d67c84354b948378051d281c3
I20260811 02:51:55.169441 14815 ts_tablet_manager.cc:1939] T 77b2803d67c84354b948378051d281c3 P 9f0b981b925841fabaeb66b5c6650494: Deleting consensus metadata
I20260811 02:51:55.169605 14500 tablet_service.cc:1467] Processing CreateTablet for tablet 3466ff551c5a420eaae8ed66b21fc0d0 (DEFAULT_TABLE table=test-table [id=46e8968786b244a7a00e91c58f61be28]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 02:51:55.169991 14500 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3466ff551c5a420eaae8ed66b21fc0d0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 02:51:55.170617 14784 tablet_bootstrap.cc:492] T 3466ff551c5a420eaae8ed66b21fc0d0 P c9b7177680454e789d9e860ba91be083: No bootstrap required, opened a new log
I20260811 02:51:55.170699 14784 ts_tablet_manager.cc:1397] T 3466ff551c5a420eaae8ed66b21fc0d0 P c9b7177680454e789d9e860ba91be083: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260811 02:51:55.171429 14784 raft_consensus.cc:348] T 3466ff551c5a420eaae8ed66b21fc0d0 P c9b7177680454e789d9e860ba91be083 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c9b7177680454e789d9e860ba91be083" member_type: VOTER last_known_addr { host: "127.12.138.193" port: 43377 } } peers { permanent_uuid: "ad6ef753e62b48c095f130de9398f2b5" member_type: VOTER last_known_addr { host: "127.12.138.194" port: 42189 } } peers { permanent_uuid: "9f0b981b925841fabaeb66b5c6650494" member_type: VOTER last_known_addr { host: "127.12.138.195" port: 41737 } }
I20260811 02:51:55.171540 14784 raft_consensus.cc:374] T 3466ff551c5a420eaae8ed66b21fc0d0 P c9b7177680454e789d9e860ba91be083 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 02:51:55.171502 14145 catalog_manager.cc:4841] TS c9b7177680454e789d9e860ba91be083 (127.12.138.193:43377): tablet 77b2803d67c84354b948378051d281c3 (table test-table [id=c73727cccb314ab9b712358bcded99d0]) successfully deleted
I20260811 02:51:55.171571 14784 raft_consensus.cc:729] T 3466ff551c5a420eaae8ed66b21fc0d0 P c9b7177680454e789d9e860ba91be083 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c9b7177680454e789d9e860ba91be083, State: Initialized, Role: FOLLOWER
I20260811 02:51:55.171684 14784 consensus_queue.cc:260] T 3466ff551c5a420eaae8ed66b21fc0d0 P c9b7177680454e789d9e860ba91be083 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c9b7177680454e789d9e860ba91be083" member_type: VOTER last_known_addr { host: "127.12.138.193" port: 43377 } } peers { permanent_uuid: "ad6ef753e62b48c095f130de9398f2b5" member_type: VOTER last_known_addr { host: "127.12.138.194" port: 42189 } } peers { permanent_uuid: "9f0b981b925841fabaeb66b5c6650494" member_type: VOTER last_known_addr { host: "127.12.138.195" port: 41737 } }
I20260811 02:51:55.171928 14784 ts_tablet_manager.cc:1428] T 3466ff551c5a420eaae8ed66b21fc0d0 P c9b7177680454e789d9e860ba91be083: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260811 02:51:55.173120 14142 catalog_manager.cc:4841] TS 9f0b981b925841fabaeb66b5c6650494 (127.12.138.195:41737): tablet 77b2803d67c84354b948378051d281c3 (table test-table [id=c73727cccb314ab9b712358bcded99d0]) successfully deleted
I20260811 02:51:55.174103 14783 tablet_bootstrap.cc:492] T 3466ff551c5a420eaae8ed66b21fc0d0 P ad6ef753e62b48c095f130de9398f2b5: Bootstrap starting.
I20260811 02:51:55.175235 14783 tablet_bootstrap.cc:654] T 3466ff551c5a420eaae8ed66b21fc0d0 P ad6ef753e62b48c095f130de9398f2b5: Neither blocks nor log segments found. Creating new log.
I20260811 02:51:55.176043 14785 tablet_bootstrap.cc:492] T 3466ff551c5a420eaae8ed66b21fc0d0 P 9f0b981b925841fabaeb66b5c6650494: Bootstrap starting.
I20260811 02:51:55.177116 14785 tablet_bootstrap.cc:654] T 3466ff551c5a420eaae8ed66b21fc0d0 P 9f0b981b925841fabaeb66b5c6650494: Neither blocks nor log segments found. Creating new log.
I20260811 02:51:55.177659 14783 tablet_bootstrap.cc:492] T 3466ff551c5a420eaae8ed66b21fc0d0 P ad6ef753e62b48c095f130de9398f2b5: No bootstrap required, opened a new log
I20260811 02:51:55.177748 14783 ts_tablet_manager.cc:1397] T 3466ff551c5a420eaae8ed66b21fc0d0 P ad6ef753e62b48c095f130de9398f2b5: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20260811 02:51:55.178174 14783 raft_consensus.cc:348] T 3466ff551c5a420eaae8ed66b21fc0d0 P ad6ef753e62b48c095f130de9398f2b5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c9b7177680454e789d9e860ba91be083" member_type: VOTER last_known_addr { host: "127.12.138.193" port: 43377 } } peers { permanent_uuid: "ad6ef753e62b48c095f130de9398f2b5" member_type: VOTER last_known_addr { host: "127.12.138.194" port: 42189 } } peers { permanent_uuid: "9f0b981b925841fabaeb66b5c6650494" member_type: VOTER last_known_addr { host: "127.12.138.195" port: 41737 } }
I20260811 02:51:55.178289 14783 raft_consensus.cc:374] T 3466ff551c5a420eaae8ed66b21fc0d0 P ad6ef753e62b48c095f130de9398f2b5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 02:51:55.178334 14783 raft_consensus.cc:729] T 3466ff551c5a420eaae8ed66b21fc0d0 P ad6ef753e62b48c095f130de9398f2b5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ad6ef753e62b48c095f130de9398f2b5, State: Initialized, Role: FOLLOWER
I20260811 02:51:55.178483 14783 consensus_queue.cc:260] T 3466ff551c5a420eaae8ed66b21fc0d0 P ad6ef753e62b48c095f130de9398f2b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c9b7177680454e789d9e860ba91be083" member_type: VOTER last_known_addr { host: "127.12.138.193" port: 43377 } } peers { permanent_uuid: "ad6ef753e62b48c095f130de9398f2b5" member_type: VOTER last_known_addr { host: "127.12.138.194" port: 42189 } } peers { permanent_uuid: "9f0b981b925841fabaeb66b5c6650494" member_type: VOTER last_known_addr { host: "127.12.138.195" port: 41737 } }
I20260811 02:51:55.178539 14785 tablet_bootstrap.cc:492] T 3466ff551c5a420eaae8ed66b21fc0d0 P 9f0b981b925841fabaeb66b5c6650494: No bootstrap required, opened a new log
I20260811 02:51:55.178642 14785 ts_tablet_manager.cc:1397] T 3466ff551c5a420eaae8ed66b21fc0d0 P 9f0b981b925841fabaeb66b5c6650494: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.002s
I20260811 02:51:55.178748 14783 ts_tablet_manager.cc:1428] T 3466ff551c5a420eaae8ed66b21fc0d0 P ad6ef753e62b48c095f130de9398f2b5: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260811 02:51:55.179108 14785 raft_consensus.cc:348] T 3466ff551c5a420eaae8ed66b21fc0d0 P 9f0b981b925841fabaeb66b5c6650494 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c9b7177680454e789d9e860ba91be083" member_type: VOTER last_known_addr { host: "127.12.138.193" port: 43377 } } peers { permanent_uuid: "ad6ef753e62b48c095f130de9398f2b5" member_type: VOTER last_known_addr { host: "127.12.138.194" port: 42189 } } peers { permanent_uuid: "9f0b981b925841fabaeb66b5c6650494" member_type: VOTER last_known_addr { host: "127.12.138.195" port: 41737 } }
I20260811 02:51:55.179265 14785 raft_consensus.cc:374] T 3466ff551c5a420eaae8ed66b21fc0d0 P 9f0b981b925841fabaeb66b5c6650494 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 02:51:55.179308 14785 raft_consensus.cc:729] T 3466ff551c5a420eaae8ed66b21fc0d0 P 9f0b981b925841fabaeb66b5c6650494 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9f0b981b925841fabaeb66b5c6650494, State: Initialized, Role: FOLLOWER
I20260811 02:51:55.179446 14785 consensus_queue.cc:260] T 3466ff551c5a420eaae8ed66b21fc0d0 P 9f0b981b925841fabaeb66b5c6650494 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c9b7177680454e789d9e860ba91be083" member_type: VOTER last_known_addr { host: "127.12.138.193" port: 43377 } } peers { permanent_uuid: "ad6ef753e62b48c095f130de9398f2b5" member_type: VOTER last_known_addr { host: "127.12.138.194" port: 42189 } } peers { permanent_uuid: "9f0b981b925841fabaeb66b5c6650494" member_type: VOTER last_known_addr { host: "127.12.138.195" port: 41737 } }
I20260811 02:51:55.179807 14785 ts_tablet_manager.cc:1428] T 3466ff551c5a420eaae8ed66b21fc0d0 P 9f0b981b925841fabaeb66b5c6650494: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
W20260811 02:51:55.225195 14758 tablet.cc:2369] T 3466ff551c5a420eaae8ed66b21fc0d0 P 9f0b981b925841fabaeb66b5c6650494: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 02:51:55.229550 14591 tablet.cc:2369] T 3466ff551c5a420eaae8ed66b21fc0d0 P ad6ef753e62b48c095f130de9398f2b5: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 02:51:55.251510 14400 tablet.cc:2369] T 3466ff551c5a420eaae8ed66b21fc0d0 P c9b7177680454e789d9e860ba91be083: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 02:51:55.476588 14794 raft_consensus.cc:482] T 3466ff551c5a420eaae8ed66b21fc0d0 P c9b7177680454e789d9e860ba91be083 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 02:51:55.476768 14794 raft_consensus.cc:504] T 3466ff551c5a420eaae8ed66b21fc0d0 P c9b7177680454e789d9e860ba91be083 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c9b7177680454e789d9e860ba91be083" member_type: VOTER last_known_addr { host: "127.12.138.193" port: 43377 } } peers { permanent_uuid: "ad6ef753e62b48c095f130de9398f2b5" member_type: VOTER last_known_addr { host: "127.12.138.194" port: 42189 } } peers { permanent_uuid: "9f0b981b925841fabaeb66b5c6650494" member_type: VOTER last_known_addr { host: "127.12.138.195" port: 41737 } }
I20260811 02:51:55.477967 14794 leader_election.cc:288] T 3466ff551c5a420eaae8ed66b21fc0d0 P c9b7177680454e789d9e860ba91be083 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ad6ef753e62b48c095f130de9398f2b5 (127.12.138.194:42189), 9f0b981b925841fabaeb66b5c6650494 (127.12.138.195:41737)
I20260811 02:51:55.482267 14823 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:51:55.478247 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.138.194:42189 (local address 127.12.138.193:48541)
0811 02:51:55.478526 (+ 279us) negotiation.cc:107] Waiting for socket to connect
0811 02:51:55.478552 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0811 02:51:55.478657 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:51:55.478963 (+ 306us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:51:55.478965 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:51:55.478978 (+ 13us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 02:51:55.479125 (+ 147us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:51:55.479137 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:51:55.480054 (+ 917us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:51:55.480057 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:51:55.481013 (+ 956us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:51:55.481022 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:51:55.481832 (+ 810us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:51:55.481837 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:51:55.481943 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:51:55.481978 (+ 35us) client_negotiation.cc:769] Sending connection context
0811 02:51:55.482058 (+ 80us) client_negotiation.cc:240] Negotiation successful
0811 02:51:55.482128 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":196,"thread_start_us":103,"threads_started":1}
I20260811 02:51:55.482300 14780 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:51:55.478403 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.138.193:48541 (local address 127.12.138.194:42189)
0811 02:51:55.478545 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0811 02:51:55.478551 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 02:51:55.478645 (+ 94us) server_negotiation.cc:407] Connection header received
0811 02:51:55.478771 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:51:55.478775 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:51:55.478847 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:51:55.478944 (+ 97us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 02:51:55.479265 (+ 321us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:51:55.479937 (+ 672us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:51:55.481171 (+ 1234us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:51:55.481674 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:51:55.481769 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:51:55.481811 (+ 42us) server_negotiation.cc:1035] Waiting for connection context
0811 02:51:55.482089 (+ 278us) server_negotiation.cc:299] Negotiation successful
0811 02:51:55.482158 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260811 02:51:55.482550 14778 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:51:55.478332 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.138.193:48963 (local address 127.12.138.195:41737)
0811 02:51:55.478490 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0811 02:51:55.478496 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 02:51:55.479090 (+ 594us) server_negotiation.cc:407] Connection header received
0811 02:51:55.479215 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:51:55.479221 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:51:55.479285 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:51:55.479381 (+ 96us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 02:51:55.479653 (+ 272us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:51:55.480270 (+ 617us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:51:55.481456 (+ 1186us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:51:55.481942 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:51:55.482072 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:51:55.482120 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0811 02:51:55.482347 (+ 227us) server_negotiation.cc:299] Negotiation successful
0811 02:51:55.482444 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260811 02:51:55.482560 14824 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:51:55.478250 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.138.195:41737 (local address 127.12.138.193:48963)
0811 02:51:55.478983 (+ 733us) negotiation.cc:107] Waiting for socket to connect
0811 02:51:55.478997 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 02:51:55.479099 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:51:55.479389 (+ 290us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:51:55.479393 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:51:55.479410 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 02:51:55.479533 (+ 123us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:51:55.479540 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:51:55.480424 (+ 884us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:51:55.480426 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:51:55.481340 (+ 914us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:51:55.481351 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:51:55.482089 (+ 738us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:51:55.482093 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:51:55.482198 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:51:55.482222 (+ 24us) client_negotiation.cc:769] Sending connection context
0811 02:51:55.482302 (+ 80us) client_negotiation.cc:240] Negotiation successful
0811 02:51:55.482373 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":671,"mutex_wait_us":313,"thread_start_us":115,"threads_started":1}
I20260811 02:51:55.483147 14527 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3466ff551c5a420eaae8ed66b21fc0d0" candidate_uuid: "c9b7177680454e789d9e860ba91be083" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ad6ef753e62b48c095f130de9398f2b5" is_pre_election: true
I20260811 02:51:55.483283 14527 raft_consensus.cc:2454] T 3466ff551c5a420eaae8ed66b21fc0d0 P ad6ef753e62b48c095f130de9398f2b5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c9b7177680454e789d9e860ba91be083 in term 0.
I20260811 02:51:55.483266 14702 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3466ff551c5a420eaae8ed66b21fc0d0" candidate_uuid: "c9b7177680454e789d9e860ba91be083" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9f0b981b925841fabaeb66b5c6650494" is_pre_election: true
I20260811 02:51:55.483565 14702 raft_consensus.cc:2454] T 3466ff551c5a420eaae8ed66b21fc0d0 P 9f0b981b925841fabaeb66b5c6650494 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c9b7177680454e789d9e860ba91be083 in term 0.
I20260811 02:51:55.483721 14263 leader_election.cc:302] T 3466ff551c5a420eaae8ed66b21fc0d0 P c9b7177680454e789d9e860ba91be083 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ad6ef753e62b48c095f130de9398f2b5, c9b7177680454e789d9e860ba91be083; no voters:
I20260811 02:51:55.484018 14794 raft_consensus.cc:2781] T 3466ff551c5a420eaae8ed66b21fc0d0 P c9b7177680454e789d9e860ba91be083 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 02:51:55.484090 14794 raft_consensus.cc:482] T 3466ff551c5a420eaae8ed66b21fc0d0 P c9b7177680454e789d9e860ba91be083 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 02:51:55.484130 14794 raft_consensus.cc:3037] T 3466ff551c5a420eaae8ed66b21fc0d0 P c9b7177680454e789d9e860ba91be083 [term 0 FOLLOWER]: Advancing to term 1
I20260811 02:51:55.485172 14794 raft_consensus.cc:504] T 3466ff551c5a420eaae8ed66b21fc0d0 P c9b7177680454e789d9e860ba91be083 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c9b7177680454e789d9e860ba91be083" member_type: VOTER last_known_addr { host: "127.12.138.193" port: 43377 } } peers { permanent_uuid: "ad6ef753e62b48c095f130de9398f2b5" member_type: VOTER last_known_addr { host: "127.12.138.194" port: 42189 } } peers { permanent_uuid: "9f0b981b925841fabaeb66b5c6650494" member_type: VOTER last_known_addr { host: "127.12.138.195" port: 41737 } }
I20260811 02:51:55.485526 14794 leader_election.cc:288] T 3466ff551c5a420eaae8ed66b21fc0d0 P c9b7177680454e789d9e860ba91be083 [CANDIDATE]: Term 1 election: Requested vote from peers ad6ef753e62b48c095f130de9398f2b5 (127.12.138.194:42189), 9f0b981b925841fabaeb66b5c6650494 (127.12.138.195:41737)
I20260811 02:51:55.485867 14702 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3466ff551c5a420eaae8ed66b21fc0d0" candidate_uuid: "c9b7177680454e789d9e860ba91be083" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9f0b981b925841fabaeb66b5c6650494"
I20260811 02:51:55.485852 14527 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3466ff551c5a420eaae8ed66b21fc0d0" candidate_uuid: "c9b7177680454e789d9e860ba91be083" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ad6ef753e62b48c095f130de9398f2b5"
I20260811 02:51:55.485993 14527 raft_consensus.cc:3037] T 3466ff551c5a420eaae8ed66b21fc0d0 P ad6ef753e62b48c095f130de9398f2b5 [term 0 FOLLOWER]: Advancing to term 1
I20260811 02:51:55.486002 14702 raft_consensus.cc:3037] T 3466ff551c5a420eaae8ed66b21fc0d0 P 9f0b981b925841fabaeb66b5c6650494 [term 0 FOLLOWER]: Advancing to term 1
I20260811 02:51:55.487239 14702 raft_consensus.cc:2454] T 3466ff551c5a420eaae8ed66b21fc0d0 P 9f0b981b925841fabaeb66b5c6650494 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c9b7177680454e789d9e860ba91be083 in term 1.
I20260811 02:51:55.487239 14527 raft_consensus.cc:2454] T 3466ff551c5a420eaae8ed66b21fc0d0 P ad6ef753e62b48c095f130de9398f2b5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c9b7177680454e789d9e860ba91be083 in term 1.
I20260811 02:51:55.487545 14264 leader_election.cc:302] T 3466ff551c5a420eaae8ed66b21fc0d0 P c9b7177680454e789d9e860ba91be083 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9f0b981b925841fabaeb66b5c6650494, c9b7177680454e789d9e860ba91be083; no voters:
I20260811 02:51:55.487809 14794 raft_consensus.cc:2781] T 3466ff551c5a420eaae8ed66b21fc0d0 P c9b7177680454e789d9e860ba91be083 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 02:51:55.488051 14794 raft_consensus.cc:686] T 3466ff551c5a420eaae8ed66b21fc0d0 P c9b7177680454e789d9e860ba91be083 [term 1 LEADER]: Becoming Leader. State: Replica: c9b7177680454e789d9e860ba91be083, State: Running, Role: LEADER
I20260811 02:51:55.488314 14794 consensus_queue.cc:237] T 3466ff551c5a420eaae8ed66b21fc0d0 P c9b7177680454e789d9e860ba91be083 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c9b7177680454e789d9e860ba91be083" member_type: VOTER last_known_addr { host: "127.12.138.193" port: 43377 } } peers { permanent_uuid: "ad6ef753e62b48c095f130de9398f2b5" member_type: VOTER last_known_addr { host: "127.12.138.194" port: 42189 } } peers { permanent_uuid: "9f0b981b925841fabaeb66b5c6650494" member_type: VOTER last_known_addr { host: "127.12.138.195" port: 41737 } }
I20260811 02:51:55.490842 14158 catalog_manager.cc:5495] T 3466ff551c5a420eaae8ed66b21fc0d0 P c9b7177680454e789d9e860ba91be083 reported cstate change: term changed from 0 to 1, leader changed from <none> to c9b7177680454e789d9e860ba91be083 (127.12.138.193). New cstate: current_term: 1 leader_uuid: "c9b7177680454e789d9e860ba91be083" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c9b7177680454e789d9e860ba91be083" member_type: VOTER last_known_addr { host: "127.12.138.193" port: 43377 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ad6ef753e62b48c095f130de9398f2b5" member_type: VOTER last_known_addr { host: "127.12.138.194" port: 42189 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9f0b981b925841fabaeb66b5c6650494" member_type: VOTER last_known_addr { host: "127.12.138.195" port: 41737 } health_report { overall_health: UNKNOWN } } }
Aug 11 02:51:55 dist-test-slave-9gcw krb5kdc[14106](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416714, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.138.254@KRBTEST.COM for kudu/127.12.138.254@KRBTEST.COM
I20260811 02:51:55.563977 14835 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:51:55.555085 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59750 (local address 127.12.138.254:45929)
0811 02:51:55.555380 (+ 295us) server_negotiation.cc:206] Beginning negotiation
0811 02:51:55.555391 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 02:51:55.555474 (+ 83us) server_negotiation.cc:407] Connection header received
0811 02:51:55.555657 (+ 183us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:51:55.555660 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:51:55.555706 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:51:55.555811 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:51:55.556488 (+ 677us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:51:55.557041 (+ 553us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:51:55.557927 (+ 886us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:51:55.558166 (+ 239us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:51:55.558278 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:51:55.560681 (+ 2403us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:51:55.560691 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:51:55.560695 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:51:55.560714 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:51:55.562099 (+ 1385us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:51:55.562602 (+ 503us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:51:55.562607 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:51:55.562609 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:51:55.562696 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:51:55.562984 (+ 288us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:51:55.562989 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:51:55.562991 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:51:55.563227 (+ 236us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:51:55.563340 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0811 02:51:55.563656 (+ 316us) server_negotiation.cc:299] Negotiation successful
0811 02:51:55.563812 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":93,"server-negotiator.queue_time_us":227,"thread_start_us":144,"threads_started":1}
I20260811 02:51:55.564021 14834 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:51:55.555038 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.138.254:45929 (local address 127.0.0.1:59750)
0811 02:51:55.555367 (+ 329us) negotiation.cc:107] Waiting for socket to connect
0811 02:51:55.555384 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 02:51:55.555541 (+ 157us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:51:55.555827 (+ 286us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:51:55.555830 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:51:55.556262 (+ 432us) client_negotiation.cc:189] Negotiated authn=SASL
0811 02:51:55.556362 (+ 100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:51:55.556369 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:51:55.557212 (+ 843us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:51:55.557215 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:51:55.557804 (+ 589us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:51:55.557812 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:51:55.558290 (+ 478us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:51:55.558293 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:51:55.558421 (+ 128us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:51:55.559090 (+ 669us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 02:51:55.559102 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0811 02:51:55.560550 (+ 1448us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 02:51:55.562247 (+ 1697us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:51:55.562258 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:51:55.562264 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:51:55.562498 (+ 234us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:51:55.562789 (+ 291us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:51:55.562792 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:51:55.562794 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:51:55.562881 (+ 87us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:51:55.563340 (+ 459us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 02:51:55.563348 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 02:51:55.563465 (+ 117us) client_negotiation.cc:769] Sending connection context
0811 02:51:55.563584 (+ 119us) client_negotiation.cc:240] Negotiation successful
0811 02:51:55.563759 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":198,"mutex_wait_us":3,"thread_start_us":90,"threads_started":1}
I20260811 02:51:55.568667 14158 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.12.138.254@KRBTEST.COM'} at 127.0.0.1:59750:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260811 02:51:55.569476 14158 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 02:51:55.574796 14500 tablet_service.cc:1467] Processing CreateTablet for tablet efa68f73790b47098c64137ea7eea566 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=d03eccc2559d45e4abb35c69ddd6b251]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260811 02:51:55.574796 14318 tablet_service.cc:1467] Processing CreateTablet for tablet efa68f73790b47098c64137ea7eea566 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=d03eccc2559d45e4abb35c69ddd6b251]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260811 02:51:55.574967 14679 tablet_service.cc:1467] Processing CreateTablet for tablet efa68f73790b47098c64137ea7eea566 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=d03eccc2559d45e4abb35c69ddd6b251]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260811 02:51:55.575222 14500 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet efa68f73790b47098c64137ea7eea566. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 02:51:55.575222 14318 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet efa68f73790b47098c64137ea7eea566. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 02:51:55.575418 14679 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet efa68f73790b47098c64137ea7eea566. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 02:51:55.577769 14785 tablet_bootstrap.cc:492] T efa68f73790b47098c64137ea7eea566 P 9f0b981b925841fabaeb66b5c6650494: Bootstrap starting.
I20260811 02:51:55.578297 14783 tablet_bootstrap.cc:492] T efa68f73790b47098c64137ea7eea566 P ad6ef753e62b48c095f130de9398f2b5: Bootstrap starting.
I20260811 02:51:55.578297 14784 tablet_bootstrap.cc:492] T efa68f73790b47098c64137ea7eea566 P c9b7177680454e789d9e860ba91be083: Bootstrap starting.
I20260811 02:51:55.578796 14785 tablet_bootstrap.cc:654] T efa68f73790b47098c64137ea7eea566 P 9f0b981b925841fabaeb66b5c6650494: Neither blocks nor log segments found. Creating new log.
I20260811 02:51:55.579269 14784 tablet_bootstrap.cc:654] T efa68f73790b47098c64137ea7eea566 P c9b7177680454e789d9e860ba91be083: Neither blocks nor log segments found. Creating new log.
I20260811 02:51:55.579412 14783 tablet_bootstrap.cc:654] T efa68f73790b47098c64137ea7eea566 P ad6ef753e62b48c095f130de9398f2b5: Neither blocks nor log segments found. Creating new log.
I20260811 02:51:55.580188 14785 tablet_bootstrap.cc:492] T efa68f73790b47098c64137ea7eea566 P 9f0b981b925841fabaeb66b5c6650494: No bootstrap required, opened a new log
I20260811 02:51:55.580293 14785 ts_tablet_manager.cc:1397] T efa68f73790b47098c64137ea7eea566 P 9f0b981b925841fabaeb66b5c6650494: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.002s
I20260811 02:51:55.580761 14785 raft_consensus.cc:348] T efa68f73790b47098c64137ea7eea566 P 9f0b981b925841fabaeb66b5c6650494 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c9b7177680454e789d9e860ba91be083" member_type: VOTER last_known_addr { host: "127.12.138.193" port: 43377 } } peers { permanent_uuid: "ad6ef753e62b48c095f130de9398f2b5" member_type: VOTER last_known_addr { host: "127.12.138.194" port: 42189 } } peers { permanent_uuid: "9f0b981b925841fabaeb66b5c6650494" member_type: VOTER last_known_addr { host: "127.12.138.195" port: 41737 } }
I20260811 02:51:55.580919 14785 raft_consensus.cc:374] T efa68f73790b47098c64137ea7eea566 P 9f0b981b925841fabaeb66b5c6650494 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 02:51:55.580963 14785 raft_consensus.cc:729] T efa68f73790b47098c64137ea7eea566 P 9f0b981b925841fabaeb66b5c6650494 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9f0b981b925841fabaeb66b5c6650494, State: Initialized, Role: FOLLOWER
I20260811 02:51:55.581065 14784 tablet_bootstrap.cc:492] T efa68f73790b47098c64137ea7eea566 P c9b7177680454e789d9e860ba91be083: No bootstrap required, opened a new log
I20260811 02:51:55.581065 14783 tablet_bootstrap.cc:492] T efa68f73790b47098c64137ea7eea566 P ad6ef753e62b48c095f130de9398f2b5: No bootstrap required, opened a new log
I20260811 02:51:55.581182 14783 ts_tablet_manager.cc:1397] T efa68f73790b47098c64137ea7eea566 P ad6ef753e62b48c095f130de9398f2b5: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260811 02:51:55.581113 14785 consensus_queue.cc:260] T efa68f73790b47098c64137ea7eea566 P 9f0b981b925841fabaeb66b5c6650494 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c9b7177680454e789d9e860ba91be083" member_type: VOTER last_known_addr { host: "127.12.138.193" port: 43377 } } peers { permanent_uuid: "ad6ef753e62b48c095f130de9398f2b5" member_type: VOTER last_known_addr { host: "127.12.138.194" port: 42189 } } peers { permanent_uuid: "9f0b981b925841fabaeb66b5c6650494" member_type: VOTER last_known_addr { host: "127.12.138.195" port: 41737 } }
I20260811 02:51:55.581218 14784 ts_tablet_manager.cc:1397] T efa68f73790b47098c64137ea7eea566 P c9b7177680454e789d9e860ba91be083: Time spent bootstrapping tablet: real 0.003s user 0.003s sys 0.000s
I20260811 02:51:55.581393 14807 tablet_replica.cc:439] T efa68f73790b47098c64137ea7eea566 P 9f0b981b925841fabaeb66b5c6650494: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c9b7177680454e789d9e860ba91be083" member_type: VOTER last_known_addr { host: "127.12.138.193" port: 43377 } } peers { permanent_uuid: "ad6ef753e62b48c095f130de9398f2b5" member_type: VOTER last_known_addr { host: "127.12.138.194" port: 42189 } } peers { permanent_uuid: "9f0b981b925841fabaeb66b5c6650494" member_type: VOTER last_known_addr { host: "127.12.138.195" port: 41737 } } }
I20260811 02:51:55.581534 14807 tablet_replica.cc:442] T efa68f73790b47098c64137ea7eea566 P 9f0b981b925841fabaeb66b5c6650494: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 02:51:55.581771 14785 ts_tablet_manager.cc:1428] T efa68f73790b47098c64137ea7eea566 P 9f0b981b925841fabaeb66b5c6650494: Time spent starting tablet: real 0.001s user 0.001s sys 0.001s
I20260811 02:51:55.581861 14783 raft_consensus.cc:348] T efa68f73790b47098c64137ea7eea566 P ad6ef753e62b48c095f130de9398f2b5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c9b7177680454e789d9e860ba91be083" member_type: VOTER last_known_addr { host: "127.12.138.193" port: 43377 } } peers { permanent_uuid: "ad6ef753e62b48c095f130de9398f2b5" member_type: VOTER last_known_addr { host: "127.12.138.194" port: 42189 } } peers { permanent_uuid: "9f0b981b925841fabaeb66b5c6650494" member_type: VOTER last_known_addr { host: "127.12.138.195" port: 41737 } }
I20260811 02:51:55.581867 14784 raft_consensus.cc:348] T efa68f73790b47098c64137ea7eea566 P c9b7177680454e789d9e860ba91be083 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c9b7177680454e789d9e860ba91be083" member_type: VOTER last_known_addr { host: "127.12.138.193" port: 43377 } } peers { permanent_uuid: "ad6ef753e62b48c095f130de9398f2b5" member_type: VOTER last_known_addr { host: "127.12.138.194" port: 42189 } } peers { permanent_uuid: "9f0b981b925841fabaeb66b5c6650494" member_type: VOTER last_known_addr { host: "127.12.138.195" port: 41737 } }
I20260811 02:51:55.581981 14784 raft_consensus.cc:374] T efa68f73790b47098c64137ea7eea566 P c9b7177680454e789d9e860ba91be083 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 02:51:55.581981 14783 raft_consensus.cc:374] T efa68f73790b47098c64137ea7eea566 P ad6ef753e62b48c095f130de9398f2b5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 02:51:55.582017 14783 raft_consensus.cc:729] T efa68f73790b47098c64137ea7eea566 P ad6ef753e62b48c095f130de9398f2b5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ad6ef753e62b48c095f130de9398f2b5, State: Initialized, Role: FOLLOWER
I20260811 02:51:55.582017 14784 raft_consensus.cc:729] T efa68f73790b47098c64137ea7eea566 P c9b7177680454e789d9e860ba91be083 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c9b7177680454e789d9e860ba91be083, State: Initialized, Role: FOLLOWER
I20260811 02:51:55.582150 14783 consensus_queue.cc:260] T efa68f73790b47098c64137ea7eea566 P ad6ef753e62b48c095f130de9398f2b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c9b7177680454e789d9e860ba91be083" member_type: VOTER last_known_addr { host: "127.12.138.193" port: 43377 } } peers { permanent_uuid: "ad6ef753e62b48c095f130de9398f2b5" member_type: VOTER last_known_addr { host: "127.12.138.194" port: 42189 } } peers { permanent_uuid: "9f0b981b925841fabaeb66b5c6650494" member_type: VOTER last_known_addr { host: "127.12.138.195" port: 41737 } }
I20260811 02:51:55.582150 14784 consensus_queue.cc:260] T efa68f73790b47098c64137ea7eea566 P c9b7177680454e789d9e860ba91be083 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c9b7177680454e789d9e860ba91be083" member_type: VOTER last_known_addr { host: "127.12.138.193" port: 43377 } } peers { permanent_uuid: "ad6ef753e62b48c095f130de9398f2b5" member_type: VOTER last_known_addr { host: "127.12.138.194" port: 42189 } } peers { permanent_uuid: "9f0b981b925841fabaeb66b5c6650494" member_type: VOTER last_known_addr { host: "127.12.138.195" port: 41737 } }
I20260811 02:51:55.582423 14784 ts_tablet_manager.cc:1428] T efa68f73790b47098c64137ea7eea566 P c9b7177680454e789d9e860ba91be083: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260811 02:51:55.582404 14793 tablet_replica.cc:439] T efa68f73790b47098c64137ea7eea566 P ad6ef753e62b48c095f130de9398f2b5: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c9b7177680454e789d9e860ba91be083" member_type: VOTER last_known_addr { host: "127.12.138.193" port: 43377 } } peers { permanent_uuid: "ad6ef753e62b48c095f130de9398f2b5" member_type: VOTER last_known_addr { host: "127.12.138.194" port: 42189 } } peers { permanent_uuid: "9f0b981b925841fabaeb66b5c6650494" member_type: VOTER last_known_addr { host: "127.12.138.195" port: 41737 } } }
I20260811 02:51:55.582553 14793 tablet_replica.cc:442] T efa68f73790b47098c64137ea7eea566 P ad6ef753e62b48c095f130de9398f2b5: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 02:51:55.582423 14783 ts_tablet_manager.cc:1428] T efa68f73790b47098c64137ea7eea566 P ad6ef753e62b48c095f130de9398f2b5: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260811 02:51:55.582794 14794 tablet_replica.cc:439] T efa68f73790b47098c64137ea7eea566 P c9b7177680454e789d9e860ba91be083: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c9b7177680454e789d9e860ba91be083" member_type: VOTER last_known_addr { host: "127.12.138.193" port: 43377 } } peers { permanent_uuid: "ad6ef753e62b48c095f130de9398f2b5" member_type: VOTER last_known_addr { host: "127.12.138.194" port: 42189 } } peers { permanent_uuid: "9f0b981b925841fabaeb66b5c6650494" member_type: VOTER last_known_addr { host: "127.12.138.195" port: 41737 } } }
I20260811 02:51:55.582935 14794 tablet_replica.cc:442] T efa68f73790b47098c64137ea7eea566 P c9b7177680454e789d9e860ba91be083: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 02:51:55.825086 14794 raft_consensus.cc:482] T efa68f73790b47098c64137ea7eea566 P c9b7177680454e789d9e860ba91be083 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 02:51:55.825258 14794 raft_consensus.cc:504] T efa68f73790b47098c64137ea7eea566 P c9b7177680454e789d9e860ba91be083 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c9b7177680454e789d9e860ba91be083" member_type: VOTER last_known_addr { host: "127.12.138.193" port: 43377 } } peers { permanent_uuid: "ad6ef753e62b48c095f130de9398f2b5" member_type: VOTER last_known_addr { host: "127.12.138.194" port: 42189 } } peers { permanent_uuid: "9f0b981b925841fabaeb66b5c6650494" member_type: VOTER last_known_addr { host: "127.12.138.195" port: 41737 } }
I20260811 02:51:55.825793 14794 leader_election.cc:288] T efa68f73790b47098c64137ea7eea566 P c9b7177680454e789d9e860ba91be083 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ad6ef753e62b48c095f130de9398f2b5 (127.12.138.194:42189), 9f0b981b925841fabaeb66b5c6650494 (127.12.138.195:41737)
I20260811 02:51:55.826330 14702 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "efa68f73790b47098c64137ea7eea566" candidate_uuid: "c9b7177680454e789d9e860ba91be083" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9f0b981b925841fabaeb66b5c6650494" is_pre_election: true
I20260811 02:51:55.826345 14527 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "efa68f73790b47098c64137ea7eea566" candidate_uuid: "c9b7177680454e789d9e860ba91be083" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ad6ef753e62b48c095f130de9398f2b5" is_pre_election: true
I20260811 02:51:55.826507 14702 raft_consensus.cc:2454] T efa68f73790b47098c64137ea7eea566 P 9f0b981b925841fabaeb66b5c6650494 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c9b7177680454e789d9e860ba91be083 in term 0.
I20260811 02:51:55.826516 14527 raft_consensus.cc:2454] T efa68f73790b47098c64137ea7eea566 P ad6ef753e62b48c095f130de9398f2b5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c9b7177680454e789d9e860ba91be083 in term 0.
I20260811 02:51:55.826850 14264 leader_election.cc:302] T efa68f73790b47098c64137ea7eea566 P c9b7177680454e789d9e860ba91be083 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9f0b981b925841fabaeb66b5c6650494, c9b7177680454e789d9e860ba91be083; no voters:
I20260811 02:51:55.827051 14794 raft_consensus.cc:2781] T efa68f73790b47098c64137ea7eea566 P c9b7177680454e789d9e860ba91be083 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 02:51:55.827121 14794 raft_consensus.cc:482] T efa68f73790b47098c64137ea7eea566 P c9b7177680454e789d9e860ba91be083 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 02:51:55.827193 14794 raft_consensus.cc:3037] T efa68f73790b47098c64137ea7eea566 P c9b7177680454e789d9e860ba91be083 [term 0 FOLLOWER]: Advancing to term 1
I20260811 02:51:55.828193 14794 raft_consensus.cc:504] T efa68f73790b47098c64137ea7eea566 P c9b7177680454e789d9e860ba91be083 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c9b7177680454e789d9e860ba91be083" member_type: VOTER last_known_addr { host: "127.12.138.193" port: 43377 } } peers { permanent_uuid: "ad6ef753e62b48c095f130de9398f2b5" member_type: VOTER last_known_addr { host: "127.12.138.194" port: 42189 } } peers { permanent_uuid: "9f0b981b925841fabaeb66b5c6650494" member_type: VOTER last_known_addr { host: "127.12.138.195" port: 41737 } }
I20260811 02:51:55.828549 14794 leader_election.cc:288] T efa68f73790b47098c64137ea7eea566 P c9b7177680454e789d9e860ba91be083 [CANDIDATE]: Term 1 election: Requested vote from peers ad6ef753e62b48c095f130de9398f2b5 (127.12.138.194:42189), 9f0b981b925841fabaeb66b5c6650494 (127.12.138.195:41737)
I20260811 02:51:55.828886 14527 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "efa68f73790b47098c64137ea7eea566" candidate_uuid: "c9b7177680454e789d9e860ba91be083" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ad6ef753e62b48c095f130de9398f2b5"
I20260811 02:51:55.828903 14702 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "efa68f73790b47098c64137ea7eea566" candidate_uuid: "c9b7177680454e789d9e860ba91be083" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9f0b981b925841fabaeb66b5c6650494"
I20260811 02:51:55.829008 14527 raft_consensus.cc:3037] T efa68f73790b47098c64137ea7eea566 P ad6ef753e62b48c095f130de9398f2b5 [term 0 FOLLOWER]: Advancing to term 1
I20260811 02:51:55.829041 14702 raft_consensus.cc:3037] T efa68f73790b47098c64137ea7eea566 P 9f0b981b925841fabaeb66b5c6650494 [term 0 FOLLOWER]: Advancing to term 1
I20260811 02:51:55.829944 14527 raft_consensus.cc:2454] T efa68f73790b47098c64137ea7eea566 P ad6ef753e62b48c095f130de9398f2b5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c9b7177680454e789d9e860ba91be083 in term 1.
I20260811 02:51:55.829974 14702 raft_consensus.cc:2454] T efa68f73790b47098c64137ea7eea566 P 9f0b981b925841fabaeb66b5c6650494 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c9b7177680454e789d9e860ba91be083 in term 1.
I20260811 02:51:55.830220 14263 leader_election.cc:302] T efa68f73790b47098c64137ea7eea566 P c9b7177680454e789d9e860ba91be083 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ad6ef753e62b48c095f130de9398f2b5, c9b7177680454e789d9e860ba91be083; no voters:
I20260811 02:51:55.830404 14794 raft_consensus.cc:2781] T efa68f73790b47098c64137ea7eea566 P c9b7177680454e789d9e860ba91be083 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 02:51:55.830498 14794 raft_consensus.cc:686] T efa68f73790b47098c64137ea7eea566 P c9b7177680454e789d9e860ba91be083 [term 1 LEADER]: Becoming Leader. State: Replica: c9b7177680454e789d9e860ba91be083, State: Running, Role: LEADER
I20260811 02:51:55.830634 14794 consensus_queue.cc:237] T efa68f73790b47098c64137ea7eea566 P c9b7177680454e789d9e860ba91be083 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c9b7177680454e789d9e860ba91be083" member_type: VOTER last_known_addr { host: "127.12.138.193" port: 43377 } } peers { permanent_uuid: "ad6ef753e62b48c095f130de9398f2b5" member_type: VOTER last_known_addr { host: "127.12.138.194" port: 42189 } } peers { permanent_uuid: "9f0b981b925841fabaeb66b5c6650494" member_type: VOTER last_known_addr { host: "127.12.138.195" port: 41737 } }
I20260811 02:51:55.831247 14825 tablet_replica.cc:439] T efa68f73790b47098c64137ea7eea566 P c9b7177680454e789d9e860ba91be083: TxnStatusTablet state changed. Reason: New leader c9b7177680454e789d9e860ba91be083. Latest consensus state: current_term: 1 leader_uuid: "c9b7177680454e789d9e860ba91be083" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c9b7177680454e789d9e860ba91be083" member_type: VOTER last_known_addr { host: "127.12.138.193" port: 43377 } } peers { permanent_uuid: "ad6ef753e62b48c095f130de9398f2b5" member_type: VOTER last_known_addr { host: "127.12.138.194" port: 42189 } } peers { permanent_uuid: "9f0b981b925841fabaeb66b5c6650494" member_type: VOTER last_known_addr { host: "127.12.138.195" port: 41737 } } }
I20260811 02:51:55.831383 14825 tablet_replica.cc:442] T efa68f73790b47098c64137ea7eea566 P c9b7177680454e789d9e860ba91be083: This TxnStatusTablet replica's current role is: LEADER
I20260811 02:51:55.831616 14840 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260811 02:51:55.831816 14840 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260811 02:51:55.832366 14158 catalog_manager.cc:5495] T efa68f73790b47098c64137ea7eea566 P c9b7177680454e789d9e860ba91be083 reported cstate change: term changed from 0 to 1, leader changed from <none> to c9b7177680454e789d9e860ba91be083 (127.12.138.193). New cstate: current_term: 1 leader_uuid: "c9b7177680454e789d9e860ba91be083" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c9b7177680454e789d9e860ba91be083" member_type: VOTER last_known_addr { host: "127.12.138.193" port: 43377 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ad6ef753e62b48c095f130de9398f2b5" member_type: VOTER last_known_addr { host: "127.12.138.194" port: 42189 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9f0b981b925841fabaeb66b5c6650494" member_type: VOTER last_known_addr { host: "127.12.138.195" port: 41737 } health_report { overall_health: UNKNOWN } } }
I20260811 02:51:55.883613 14834 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:51:55.878882 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.138.193:43377 (local address 127.0.0.1:54594)
0811 02:51:55.879059 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0811 02:51:55.879077 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 02:51:55.879271 (+ 194us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:51:55.879865 (+ 594us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:51:55.879869 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:51:55.879877 (+ 8us) client_negotiation.cc:189] Negotiated authn=TOKEN
0811 02:51:55.880006 (+ 129us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:51:55.880013 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:51:55.880896 (+ 883us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:51:55.880903 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:51:55.881655 (+ 752us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:51:55.881665 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:51:55.882267 (+ 602us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:51:55.882271 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:51:55.882376 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:51:55.882426 (+ 50us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0811 02:51:55.883271 (+ 845us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0811 02:51:55.883290 (+ 19us) client_negotiation.cc:769] Sending connection context
0811 02:51:55.883368 (+ 78us) client_negotiation.cc:240] Negotiation successful
0811 02:51:55.883467 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":35}
I20260811 02:51:55.883643 14843 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:51:55.879078 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54594 (local address 127.12.138.193:43377)
0811 02:51:55.879417 (+ 339us) server_negotiation.cc:206] Beginning negotiation
0811 02:51:55.879424 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 02:51:55.879444 (+ 20us) server_negotiation.cc:407] Connection header received
0811 02:51:55.879533 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:51:55.879536 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:51:55.879593 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:51:55.879838 (+ 245us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 02:51:55.880131 (+ 293us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:51:55.880755 (+ 624us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:51:55.881804 (+ 1049us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:51:55.882126 (+ 322us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:51:55.882264 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:51:55.882605 (+ 341us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 02:51:55.883164 (+ 559us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 02:51:55.883301 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0811 02:51:55.883391 (+ 90us) server_negotiation.cc:299] Negotiation successful
0811 02:51:55.883451 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":257,"thread_start_us":113,"threads_started":1}
I20260811 02:51:55.885496 14825 consensus_queue.cc:1035] T 3466ff551c5a420eaae8ed66b21fc0d0 P c9b7177680454e789d9e860ba91be083 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad6ef753e62b48c095f130de9398f2b5" member_type: VOTER last_known_addr { host: "127.12.138.194" port: 42189 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 02:51:55.887439 14528 raft_consensus.cc:1261] T efa68f73790b47098c64137ea7eea566 P ad6ef753e62b48c095f130de9398f2b5 [term 1 FOLLOWER]: Refusing update from remote peer c9b7177680454e789d9e860ba91be083: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 02:51:55.887657 14702 raft_consensus.cc:1261] T efa68f73790b47098c64137ea7eea566 P 9f0b981b925841fabaeb66b5c6650494 [term 1 FOLLOWER]: Refusing update from remote peer c9b7177680454e789d9e860ba91be083: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 02:51:55.888401 14794 consensus_queue.cc:1035] T efa68f73790b47098c64137ea7eea566 P c9b7177680454e789d9e860ba91be083 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9f0b981b925841fabaeb66b5c6650494" member_type: VOTER last_known_addr { host: "127.12.138.195" port: 41737 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 02:51:55.888723 14794 consensus_queue.cc:1035] T efa68f73790b47098c64137ea7eea566 P c9b7177680454e789d9e860ba91be083 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad6ef753e62b48c095f130de9398f2b5" member_type: VOTER last_known_addr { host: "127.12.138.194" port: 42189 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 02:51:55.890419 14807 tablet_replica.cc:439] T efa68f73790b47098c64137ea7eea566 P 9f0b981b925841fabaeb66b5c6650494: TxnStatusTablet state changed. Reason: New leader c9b7177680454e789d9e860ba91be083. Latest consensus state: current_term: 1 leader_uuid: "c9b7177680454e789d9e860ba91be083" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c9b7177680454e789d9e860ba91be083" member_type: VOTER last_known_addr { host: "127.12.138.193" port: 43377 } } peers { permanent_uuid: "ad6ef753e62b48c095f130de9398f2b5" member_type: VOTER last_known_addr { host: "127.12.138.194" port: 42189 } } peers { permanent_uuid: "9f0b981b925841fabaeb66b5c6650494" member_type: VOTER last_known_addr { host: "127.12.138.195" port: 41737 } } }
I20260811 02:51:55.890559 14807 tablet_replica.cc:442] T efa68f73790b47098c64137ea7eea566 P 9f0b981b925841fabaeb66b5c6650494: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 02:51:55.891469 14793 tablet_replica.cc:439] T efa68f73790b47098c64137ea7eea566 P ad6ef753e62b48c095f130de9398f2b5: TxnStatusTablet state changed. Reason: New leader c9b7177680454e789d9e860ba91be083. Latest consensus state: current_term: 1 leader_uuid: "c9b7177680454e789d9e860ba91be083" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c9b7177680454e789d9e860ba91be083" member_type: VOTER last_known_addr { host: "127.12.138.193" port: 43377 } } peers { permanent_uuid: "ad6ef753e62b48c095f130de9398f2b5" member_type: VOTER last_known_addr { host: "127.12.138.194" port: 42189 } } peers { permanent_uuid: "9f0b981b925841fabaeb66b5c6650494" member_type: VOTER last_known_addr { host: "127.12.138.195" port: 41737 } } }
I20260811 02:51:55.891597 14793 tablet_replica.cc:442] T efa68f73790b47098c64137ea7eea566 P ad6ef753e62b48c095f130de9398f2b5: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 02:51:55.894392 14848 tablet_replica.cc:439] T efa68f73790b47098c64137ea7eea566 P c9b7177680454e789d9e860ba91be083: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "c9b7177680454e789d9e860ba91be083" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c9b7177680454e789d9e860ba91be083" member_type: VOTER last_known_addr { host: "127.12.138.193" port: 43377 } } peers { permanent_uuid: "ad6ef753e62b48c095f130de9398f2b5" member_type: VOTER last_known_addr { host: "127.12.138.194" port: 42189 } } peers { permanent_uuid: "9f0b981b925841fabaeb66b5c6650494" member_type: VOTER last_known_addr { host: "127.12.138.195" port: 41737 } } }
I20260811 02:51:55.894529 14848 tablet_replica.cc:442] T efa68f73790b47098c64137ea7eea566 P c9b7177680454e789d9e860ba91be083: This TxnStatusTablet replica's current role is: LEADER
I20260811 02:51:55.894651 14845 mvcc.cc:205] Tried to move back new op lower bound from 7317162868269133824 to 7317162868043476992. Current Snapshot: MvccSnapshot[applied={T|T < 7317162868269133824}]
I20260811 02:51:55.894999 14794 consensus_queue.cc:1035] T 3466ff551c5a420eaae8ed66b21fc0d0 P c9b7177680454e789d9e860ba91be083 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9f0b981b925841fabaeb66b5c6650494" member_type: VOTER last_known_addr { host: "127.12.138.195" port: 41737 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 02:51:55.895709 14807 tablet_replica.cc:439] T efa68f73790b47098c64137ea7eea566 P 9f0b981b925841fabaeb66b5c6650494: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "c9b7177680454e789d9e860ba91be083" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c9b7177680454e789d9e860ba91be083" member_type: VOTER last_known_addr { host: "127.12.138.193" port: 43377 } } peers { permanent_uuid: "ad6ef753e62b48c095f130de9398f2b5" member_type: VOTER last_known_addr { host: "127.12.138.194" port: 42189 } } peers { permanent_uuid: "9f0b981b925841fabaeb66b5c6650494" member_type: VOTER last_known_addr { host: "127.12.138.195" port: 41737 } } }
I20260811 02:51:55.895839 14807 tablet_replica.cc:442] T efa68f73790b47098c64137ea7eea566 P 9f0b981b925841fabaeb66b5c6650494: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 02:51:55.896442 14850 mvcc.cc:205] Tried to move back new op lower bound from 7317162868269133824 to 7317162868043476992. Current Snapshot: MvccSnapshot[applied={T|T < 7317162868269133824}]
I20260811 02:51:55.896797 14825 tablet_replica.cc:439] T efa68f73790b47098c64137ea7eea566 P c9b7177680454e789d9e860ba91be083: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "c9b7177680454e789d9e860ba91be083" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c9b7177680454e789d9e860ba91be083" member_type: VOTER last_known_addr { host: "127.12.138.193" port: 43377 } } peers { permanent_uuid: "ad6ef753e62b48c095f130de9398f2b5" member_type: VOTER last_known_addr { host: "127.12.138.194" port: 42189 } } peers { permanent_uuid: "9f0b981b925841fabaeb66b5c6650494" member_type: VOTER last_known_addr { host: "127.12.138.195" port: 41737 } } }
I20260811 02:51:55.896912 14825 tablet_replica.cc:442] T efa68f73790b47098c64137ea7eea566 P c9b7177680454e789d9e860ba91be083: This TxnStatusTablet replica's current role is: LEADER
I20260811 02:51:55.897531 14793 tablet_replica.cc:439] T efa68f73790b47098c64137ea7eea566 P ad6ef753e62b48c095f130de9398f2b5: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "c9b7177680454e789d9e860ba91be083" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c9b7177680454e789d9e860ba91be083" member_type: VOTER last_known_addr { host: "127.12.138.193" port: 43377 } } peers { permanent_uuid: "ad6ef753e62b48c095f130de9398f2b5" member_type: VOTER last_known_addr { host: "127.12.138.194" port: 42189 } } peers { permanent_uuid: "9f0b981b925841fabaeb66b5c6650494" member_type: VOTER last_known_addr { host: "127.12.138.195" port: 41737 } } }
I20260811 02:51:55.897653 14793 tablet_replica.cc:442] T efa68f73790b47098c64137ea7eea566 P ad6ef753e62b48c095f130de9398f2b5: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 02:51:55.909433 14843 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:51:55.905529 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54596 (local address 127.12.138.193:43377)
0811 02:51:55.905691 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0811 02:51:55.905696 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 02:51:55.905851 (+ 155us) server_negotiation.cc:407] Connection header received
0811 02:51:55.906004 (+ 153us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:51:55.906007 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:51:55.906051 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:51:55.906112 (+ 61us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 02:51:55.906386 (+ 274us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:51:55.906969 (+ 583us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:51:55.908186 (+ 1217us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:51:55.908466 (+ 280us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:51:55.908559 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:51:55.908887 (+ 328us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 02:51:55.908957 (+ 70us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 02:51:55.909017 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0811 02:51:55.909151 (+ 134us) server_negotiation.cc:299] Negotiation successful
0811 02:51:55.909238 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260811 02:51:55.921762 14843 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:51:55.917696 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.138.193:49427 (local address 127.12.138.193:43377)
0811 02:51:55.917836 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0811 02:51:55.917841 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 02:51:55.918095 (+ 254us) server_negotiation.cc:407] Connection header received
0811 02:51:55.918216 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:51:55.918218 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:51:55.918261 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:51:55.918329 (+ 68us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 02:51:55.918637 (+ 308us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:51:55.919265 (+ 628us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:51:55.920519 (+ 1254us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:51:55.920783 (+ 264us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:51:55.920872 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:51:55.921238 (+ 366us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 02:51:55.921315 (+ 77us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 02:51:55.921376 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0811 02:51:55.921511 (+ 135us) server_negotiation.cc:299] Negotiation successful
0811 02:51:55.921557 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260811 02:51:55.921790 14865 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:51:55.917653 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.138.193:43377 (local address 127.12.138.193:49427)
0811 02:51:55.917993 (+ 340us) negotiation.cc:107] Waiting for socket to connect
0811 02:51:55.918009 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 02:51:55.918119 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:51:55.918361 (+ 242us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:51:55.918364 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:51:55.918372 (+ 8us) client_negotiation.cc:189] Negotiated authn=TOKEN
0811 02:51:55.918529 (+ 157us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:51:55.918535 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:51:55.919375 (+ 840us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:51:55.919379 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:51:55.920390 (+ 1011us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:51:55.920401 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:51:55.920902 (+ 501us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:51:55.920906 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:51:55.921017 (+ 111us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:51:55.921071 (+ 54us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0811 02:51:55.921407 (+ 336us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0811 02:51:55.921414 (+ 7us) client_negotiation.cc:769] Sending connection context
0811 02:51:55.921484 (+ 70us) client_negotiation.cc:240] Negotiation successful
0811 02:51:55.921563 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":227,"thread_start_us":118,"threads_started":1}
I20260811 02:51:55.969854 14158 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:59746:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 02:51:55.970055 14158 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:59746:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 02:51:55.972322 14158 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 7514043a26cd4e49b389e3b8af39c2ed: Sending DeleteTablet for 3 replicas of tablet 3466ff551c5a420eaae8ed66b21fc0d0
I20260811 02:51:55.973385 14679 tablet_service.cc:1514] Processing DeleteTablet for tablet 3466ff551c5a420eaae8ed66b21fc0d0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 02:51:55 UTC) from {username='kudu', principal='kudu/127.12.138.254@KRBTEST.COM'} at 127.0.0.1:36380
I20260811 02:51:55.973538 14500 tablet_service.cc:1514] Processing DeleteTablet for tablet 3466ff551c5a420eaae8ed66b21fc0d0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 02:51:55 UTC) from {username='kudu', principal='kudu/127.12.138.254@KRBTEST.COM'} at 127.0.0.1:47394
I20260811 02:51:55.974692 14868 tablet_replica.cc:330] T 3466ff551c5a420eaae8ed66b21fc0d0 P ad6ef753e62b48c095f130de9398f2b5: stopping tablet replica
I20260811 02:51:55.974861 14868 raft_consensus.cc:2229] T 3466ff551c5a420eaae8ed66b21fc0d0 P ad6ef753e62b48c095f130de9398f2b5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 02:51:55.974989 14868 raft_consensus.cc:2258] T 3466ff551c5a420eaae8ed66b21fc0d0 P ad6ef753e62b48c095f130de9398f2b5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 02:51:55.975807 14868 ts_tablet_manager.cc:1905] T 3466ff551c5a420eaae8ed66b21fc0d0 P ad6ef753e62b48c095f130de9398f2b5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 02:51:55.979223 14868 ts_tablet_manager.cc:1918] T 3466ff551c5a420eaae8ed66b21fc0d0 P ad6ef753e62b48c095f130de9398f2b5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 02:51:55.979329 14868 log.cc:1196] T 3466ff551c5a420eaae8ed66b21fc0d0 P ad6ef753e62b48c095f130de9398f2b5: Deleting WAL directory at /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786416707705494-12843-0/minicluster-data/ts-1/wal/wals/3466ff551c5a420eaae8ed66b21fc0d0
I20260811 02:51:55.979637 14868 ts_tablet_manager.cc:1939] T 3466ff551c5a420eaae8ed66b21fc0d0 P ad6ef753e62b48c095f130de9398f2b5: Deleting consensus metadata
I20260811 02:51:55.980468 14138 catalog_manager.cc:4841] TS ad6ef753e62b48c095f130de9398f2b5 (127.12.138.194:42189): tablet 3466ff551c5a420eaae8ed66b21fc0d0 (table test-table [id=46e8968786b244a7a00e91c58f61be28]) successfully deleted
I20260811 02:51:55.980654 14318 tablet_service.cc:1514] Processing DeleteTablet for tablet 3466ff551c5a420eaae8ed66b21fc0d0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 02:51:55 UTC) from {username='kudu', principal='kudu/127.12.138.254@KRBTEST.COM'} at 127.0.0.1:54586
I20260811 02:51:55.981169 14872 tablet_replica.cc:330] T 3466ff551c5a420eaae8ed66b21fc0d0 P c9b7177680454e789d9e860ba91be083: stopping tablet replica
I20260811 02:51:55.981353 14867 tablet_replica.cc:330] T 3466ff551c5a420eaae8ed66b21fc0d0 P 9f0b981b925841fabaeb66b5c6650494: stopping tablet replica
I20260811 02:51:55.981369 14872 raft_consensus.cc:2229] T 3466ff551c5a420eaae8ed66b21fc0d0 P c9b7177680454e789d9e860ba91be083 [term 1 LEADER]: Raft consensus shutting down.
I20260811 02:51:55.981495 14867 raft_consensus.cc:2229] T 3466ff551c5a420eaae8ed66b21fc0d0 P 9f0b981b925841fabaeb66b5c6650494 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 02:51:55.981623 14867 raft_consensus.cc:2258] T 3466ff551c5a420eaae8ed66b21fc0d0 P 9f0b981b925841fabaeb66b5c6650494 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 02:51:55.981946 14872 raft_consensus.cc:2258] T 3466ff551c5a420eaae8ed66b21fc0d0 P c9b7177680454e789d9e860ba91be083 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 02:51:55.982367 14867 ts_tablet_manager.cc:1905] T 3466ff551c5a420eaae8ed66b21fc0d0 P 9f0b981b925841fabaeb66b5c6650494: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 02:51:55.982900 14872 ts_tablet_manager.cc:1905] T 3466ff551c5a420eaae8ed66b21fc0d0 P c9b7177680454e789d9e860ba91be083: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 02:51:55.985598 14867 ts_tablet_manager.cc:1918] T 3466ff551c5a420eaae8ed66b21fc0d0 P 9f0b981b925841fabaeb66b5c6650494: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 02:51:55.985709 14867 log.cc:1196] T 3466ff551c5a420eaae8ed66b21fc0d0 P 9f0b981b925841fabaeb66b5c6650494: Deleting WAL directory at /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786416707705494-12843-0/minicluster-data/ts-2/wal/wals/3466ff551c5a420eaae8ed66b21fc0d0
I20260811 02:51:55.986019 14867 ts_tablet_manager.cc:1939] T 3466ff551c5a420eaae8ed66b21fc0d0 P 9f0b981b925841fabaeb66b5c6650494: Deleting consensus metadata
I20260811 02:51:55.986276 14872 ts_tablet_manager.cc:1918] T 3466ff551c5a420eaae8ed66b21fc0d0 P c9b7177680454e789d9e860ba91be083: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 02:51:55.986415 14872 log.cc:1196] T 3466ff551c5a420eaae8ed66b21fc0d0 P c9b7177680454e789d9e860ba91be083: Deleting WAL directory at /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786416707705494-12843-0/minicluster-data/ts-0/wal/wals/3466ff551c5a420eaae8ed66b21fc0d0
I20260811 02:51:55.986790 14872 ts_tablet_manager.cc:1939] T 3466ff551c5a420eaae8ed66b21fc0d0 P c9b7177680454e789d9e860ba91be083: Deleting consensus metadata
I20260811 02:51:55.986806 14142 catalog_manager.cc:4841] TS 9f0b981b925841fabaeb66b5c6650494 (127.12.138.195:41737): tablet 3466ff551c5a420eaae8ed66b21fc0d0 (table test-table [id=46e8968786b244a7a00e91c58f61be28]) successfully deleted
I20260811 02:51:55.987692 14145 catalog_manager.cc:4841] TS c9b7177680454e789d9e860ba91be083 (127.12.138.193:43377): tablet 3466ff551c5a420eaae8ed66b21fc0d0 (table test-table [id=46e8968786b244a7a00e91c58f61be28]) successfully deleted
Aug 11 02:51:55 dist-test-slave-9gcw krb5kdc[14106](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416714, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.12.138.193@KRBTEST.COM
I20260811 02:51:56.000181 14843 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:51:55.974880 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54602 (local address 127.12.138.193:43377)
0811 02:51:55.977281 (+ 2401us) server_negotiation.cc:206] Beginning negotiation
0811 02:51:55.977289 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 02:51:55.987322 (+ 10033us) server_negotiation.cc:407] Connection header received
0811 02:51:55.987488 (+ 166us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:51:55.987493 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:51:55.987547 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:51:55.987671 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:51:55.988631 (+ 960us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:51:55.989607 (+ 976us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:51:55.991023 (+ 1416us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:51:55.991350 (+ 327us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:51:55.991465 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:51:55.995759 (+ 4294us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:51:55.995771 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:51:55.995787 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:51:55.995817 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:51:55.998082 (+ 2265us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:51:55.998620 (+ 538us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:51:55.998627 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:51:55.998629 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:51:55.998727 (+ 98us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:51:55.999025 (+ 298us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:51:55.999028 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:51:55.999030 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:51:55.999357 (+ 327us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:51:55.999582 (+ 225us) server_negotiation.cc:1035] Waiting for connection context
0811 02:51:55.999805 (+ 223us) server_negotiation.cc:299] Negotiation successful
0811 02:51:55.999947 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2295}
W20260811 02:51:56.000802 14392 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.SetFlag from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:54602
I20260811 02:51:56.015240 14835 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:51:56.003018 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59756 (local address 127.12.138.254:45929)
0811 02:51:56.003384 (+ 366us) server_negotiation.cc:206] Beginning negotiation
0811 02:51:56.003389 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 02:51:56.005485 (+ 2096us) server_negotiation.cc:407] Connection header received
0811 02:51:56.005700 (+ 215us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:51:56.005703 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:51:56.005775 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:51:56.005835 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:51:56.006868 (+ 1033us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:51:56.007418 (+ 550us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:51:56.008801 (+ 1383us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:51:56.009079 (+ 278us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:51:56.009296 (+ 217us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:51:56.011188 (+ 1892us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:51:56.011191 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:51:56.011194 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:51:56.011222 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:51:56.013059 (+ 1837us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:51:56.013922 (+ 863us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:51:56.013925 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:51:56.013927 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:51:56.014054 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:51:56.014352 (+ 298us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:51:56.014355 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:51:56.014356 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:51:56.014577 (+ 221us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:51:56.014771 (+ 194us) server_negotiation.cc:1035] Waiting for connection context
0811 02:51:56.014967 (+ 196us) server_negotiation.cc:299] Negotiation successful
0811 02:51:56.015095 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":256}
W20260811 02:51:56.015866 14158 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:59756
I20260811 02:51:56.017836 12843 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDawEwS/build/debug/bin/kudu with pid 14245
I20260811 02:51:56.028879 12843 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDawEwS/build/debug/bin/kudu with pid 14414
I20260811 02:51:56.038527 12843 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDawEwS/build/debug/bin/kudu with pid 14600
I20260811 02:51:56.048117 12843 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDawEwS/build/debug/bin/kudu with pid 14122
2026-08-11T02:51:56Z chronyd exiting
[ OK ] SecurityITest.SmokeTestAsAuthorizedUser (3383 ms)
[ RUN ] SecurityITest.TxnSmokeWithDifferentUserTypes
Loading random data
Initializing database '/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786416707705494-12843-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 02:51:56 dist-test-slave-9gcw krb5kdc[14879](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 02:51:56 dist-test-slave-9gcw krb5kdc[14879](info): set up 2 sockets
Aug 11 02:51:56 dist-test-slave-9gcw krb5kdc[14879](info): commencing operation
krb5kdc: starting...
W20260811 02:51:57.201812 12843 mini_kdc.cc:121] Time spent starting KDC: real 1.126s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786416707705494-12843-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786416707705494-12843-0/krb5kdc/test-user.keytab.
Aug 11 02:51:57 dist-test-slave-9gcw krb5kdc[14879](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416717, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T02:51:57Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T02:51:57Z Disabled control of system clock
WARNING: no policy specified for kudu/127.12.138.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.138.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.138.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786416707705494-12843-0/krb5kdc/kudu_127.12.138.254.keytab.
Entry for principal kudu/127.12.138.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786416707705494-12843-0/krb5kdc/kudu_127.12.138.254.keytab.
I20260811 02:51:57.344524 12843 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDawEwS/build/debug/bin/kudu
/tmp/dist-test-taskDawEwS/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786416707705494-12843-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786416707705494-12843-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786416707705494-12843-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786416707705494-12843-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.138.254:41515
--webserver_interface=127.12.138.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786416707705494-12843-0/krb5kdc/kudu_127.12.138.254.keytab
--principal=kudu/127.12.138.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.138.212:41525
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.12.138.254:41515
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786416707705494-12843-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786416707705494-12843-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786416707705494-12843-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 02:51:57.465386 14896 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 02:51:57.465696 14896 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 02:51:57.465742 14896 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 02:51:57.470139 14896 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 02:51:57.470207 14896 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260811 02:51:57.470227 14896 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 02:51:57.470247 14896 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 02:51:57.470265 14896 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 02:51:57.470283 14896 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 02:51:57.476001 14896 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.138.212:41525
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786416707705494-12843-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786416707705494-12843-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.12.138.254:41515
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786416707705494-12843-0/krb5kdc/kudu_127.12.138.254.keytab
--principal=kudu/127.12.138.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.138.254:41515
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786416707705494-12843-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.12.138.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14896
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786416707705494-12843-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 02:09:49 UTC on ccycloud.quasar-fsnqqc.root.comops.site
build id 4579570
I20260811 02:51:57.477109 14896 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 02:51:57.478216 14896 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 02:51:57.487330 14906 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:51:57.487414 14896 server_base.cc:1034] running on GCE node
W20260811 02:51:57.487422 14904 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:51:57.487330 14903 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:51:57.488253 14896 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 02:51:57.489511 14896 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 02:51:57.490703 14896 hybrid_clock.cc:648] HybridClock initialized: now 1786416717490669 us; error 44 us; skew 500 ppm
Aug 11 02:51:57 dist-test-slave-9gcw krb5kdc[14879](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416717, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.138.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 02:51:57.494184 14896 init.cc:377] Logged in from keytab as kudu/127.12.138.254@KRBTEST.COM (short username kudu)
I20260811 02:51:57.495316 14896 webserver.cc:460] Webserver started at http://127.12.138.254:41563/ using document root <none> and password file <none>
I20260811 02:51:57.495975 14896 fs_manager.cc:359] Metadata directory not provided
I20260811 02:51:57.496074 14896 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 02:51:57.496340 14896 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 02:51:57.498734 14896 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786416707705494-12843-0/minicluster-data/master-0/data/instance:
uuid: "60ef762b8fc246488dfb261f0b1e8420"
format_stamp: "Formatted at 2026-08-11 02:51:57 on dist-test-slave-9gcw"
server_key: "9ff369626ef6b654732bd8d96383b46a"
server_key_iv: "39b0b247ba81b617b6a4e9ca0b6bf3f4"
server_key_version: "encryptionkey@0"
I20260811 02:51:57.499377 14896 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786416707705494-12843-0/minicluster-data/master-0/wal/instance:
uuid: "60ef762b8fc246488dfb261f0b1e8420"
format_stamp: "Formatted at 2026-08-11 02:51:57 on dist-test-slave-9gcw"
server_key: "9ff369626ef6b654732bd8d96383b46a"
server_key_iv: "39b0b247ba81b617b6a4e9ca0b6bf3f4"
server_key_version: "encryptionkey@0"
I20260811 02:51:57.503798 14896 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.001s
I20260811 02:51:57.506610 14914 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 02:51:57.507923 14896 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260811 02:51:57.508039 14896 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786416707705494-12843-0/minicluster-data/master-0/data,/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786416707705494-12843-0/minicluster-data/master-0/wal
uuid: "60ef762b8fc246488dfb261f0b1e8420"
format_stamp: "Formatted at 2026-08-11 02:51:57 on dist-test-slave-9gcw"
server_key: "9ff369626ef6b654732bd8d96383b46a"
server_key_iv: "39b0b247ba81b617b6a4e9ca0b6bf3f4"
server_key_version: "encryptionkey@0"
I20260811 02:51:57.508132 14896 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786416707705494-12843-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786416707705494-12843-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786416707705494-12843-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 02:51:57.526087 14896 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 02:51:57.529423 14896 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 02:51:57.529585 14896 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 02:51:57.537984 14896 rpc_server.cc:307] RPC server started. Bound to: 127.12.138.254:41515
I20260811 02:51:57.538024 14983 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.138.254:41515 every 8 connection(s)
I20260811 02:51:57.538985 14896 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786416707705494-12843-0/minicluster-data/master-0/data/info.pb
I20260811 02:51:57.542053 14984 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 02:51:57.542449 12843 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDawEwS/build/debug/bin/kudu as pid 14896
I20260811 02:51:57.542542 12843 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786416707705494-12843-0/minicluster-data/master-0/wal/instance
I20260811 02:51:57.542819 12843 external_mini_cluster.cc:1428] Setting key b5d9434844dc9c7e5901f2f349a99e40
I20260811 02:51:57.549567 14984 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 60ef762b8fc246488dfb261f0b1e8420: Bootstrap starting.
I20260811 02:51:57.552083 14984 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 60ef762b8fc246488dfb261f0b1e8420: Neither blocks nor log segments found. Creating new log.
Aug 11 02:51:57 dist-test-slave-9gcw krb5kdc[14879](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416717, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.12.138.254@KRBTEST.COM
I20260811 02:51:57.552911 14984 log.cc:824] T 00000000000000000000000000000000 P 60ef762b8fc246488dfb261f0b1e8420: Log is configured to *not* fsync() on all Append() calls
I20260811 02:51:57.555543 14984 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 60ef762b8fc246488dfb261f0b1e8420: No bootstrap required, opened a new log
I20260811 02:51:57.559190 14989 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:51:57.544255 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50212 (local address 127.12.138.254:41515)
0811 02:51:57.544733 (+ 478us) server_negotiation.cc:206] Beginning negotiation
0811 02:51:57.544754 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0811 02:51:57.544799 (+ 45us) server_negotiation.cc:407] Connection header received
0811 02:51:57.545657 (+ 858us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:51:57.545684 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:51:57.546125 (+ 441us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:51:57.546508 (+ 383us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:51:57.547216 (+ 708us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:51:57.548760 (+ 1544us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:51:57.549690 (+ 930us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:51:57.550370 (+ 680us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:51:57.550539 (+ 169us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:51:57.553085 (+ 2546us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:51:57.553092 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:51:57.553108 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:51:57.553167 (+ 59us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:51:57.556063 (+ 2896us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:51:57.556493 (+ 430us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:51:57.556499 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:51:57.556505 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:51:57.556655 (+ 150us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:51:57.556945 (+ 290us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:51:57.556948 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:51:57.556950 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:51:57.557486 (+ 536us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:51:57.557685 (+ 199us) server_negotiation.cc:1035] Waiting for connection context
0811 02:51:57.557926 (+ 241us) server_negotiation.cc:299] Negotiation successful
0811 02:51:57.558229 (+ 303us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":318,"thread_start_us":130,"threads_started":1}
I20260811 02:51:57.559554 14984 raft_consensus.cc:348] T 00000000000000000000000000000000 P 60ef762b8fc246488dfb261f0b1e8420 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "60ef762b8fc246488dfb261f0b1e8420" member_type: VOTER last_known_addr { host: "127.12.138.254" port: 41515 } }
I20260811 02:51:57.559856 14984 raft_consensus.cc:374] T 00000000000000000000000000000000 P 60ef762b8fc246488dfb261f0b1e8420 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 02:51:57.559962 14984 raft_consensus.cc:729] T 00000000000000000000000000000000 P 60ef762b8fc246488dfb261f0b1e8420 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 60ef762b8fc246488dfb261f0b1e8420, State: Initialized, Role: FOLLOWER
I20260811 02:51:57.560433 14984 consensus_queue.cc:260] T 00000000000000000000000000000000 P 60ef762b8fc246488dfb261f0b1e8420 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "60ef762b8fc246488dfb261f0b1e8420" member_type: VOTER last_known_addr { host: "127.12.138.254" port: 41515 } }
I20260811 02:51:57.560576 14984 raft_consensus.cc:388] T 00000000000000000000000000000000 P 60ef762b8fc246488dfb261f0b1e8420 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 02:51:57.560668 14984 raft_consensus.cc:482] T 00000000000000000000000000000000 P 60ef762b8fc246488dfb261f0b1e8420 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 02:51:57.560770 14984 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 60ef762b8fc246488dfb261f0b1e8420 [term 0 FOLLOWER]: Advancing to term 1
I20260811 02:51:57.561906 14984 raft_consensus.cc:504] T 00000000000000000000000000000000 P 60ef762b8fc246488dfb261f0b1e8420 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "60ef762b8fc246488dfb261f0b1e8420" member_type: VOTER last_known_addr { host: "127.12.138.254" port: 41515 } }
I20260811 02:51:57.562250 14984 leader_election.cc:302] T 00000000000000000000000000000000 P 60ef762b8fc246488dfb261f0b1e8420 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 60ef762b8fc246488dfb261f0b1e8420; no voters:
I20260811 02:51:57.562556 14984 leader_election.cc:288] T 00000000000000000000000000000000 P 60ef762b8fc246488dfb261f0b1e8420 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 02:51:57.562683 14991 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 60ef762b8fc246488dfb261f0b1e8420 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 02:51:57.562954 14991 raft_consensus.cc:686] T 00000000000000000000000000000000 P 60ef762b8fc246488dfb261f0b1e8420 [term 1 LEADER]: Becoming Leader. State: Replica: 60ef762b8fc246488dfb261f0b1e8420, State: Running, Role: LEADER
I20260811 02:51:57.563292 14991 consensus_queue.cc:237] T 00000000000000000000000000000000 P 60ef762b8fc246488dfb261f0b1e8420 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "60ef762b8fc246488dfb261f0b1e8420" member_type: VOTER last_known_addr { host: "127.12.138.254" port: 41515 } }
I20260811 02:51:57.563730 14984 sys_catalog.cc:564] T 00000000000000000000000000000000 P 60ef762b8fc246488dfb261f0b1e8420 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 02:51:57.565204 14993 sys_catalog.cc:455] T 00000000000000000000000000000000 P 60ef762b8fc246488dfb261f0b1e8420 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "60ef762b8fc246488dfb261f0b1e8420" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "60ef762b8fc246488dfb261f0b1e8420" member_type: VOTER last_known_addr { host: "127.12.138.254" port: 41515 } } }
I20260811 02:51:57.565133 14994 sys_catalog.cc:455] T 00000000000000000000000000000000 P 60ef762b8fc246488dfb261f0b1e8420 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 60ef762b8fc246488dfb261f0b1e8420. Latest consensus state: current_term: 1 leader_uuid: "60ef762b8fc246488dfb261f0b1e8420" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "60ef762b8fc246488dfb261f0b1e8420" member_type: VOTER last_known_addr { host: "127.12.138.254" port: 41515 } } }
I20260811 02:51:57.565303 14994 sys_catalog.cc:458] T 00000000000000000000000000000000 P 60ef762b8fc246488dfb261f0b1e8420 [sys.catalog]: This master's current role is: LEADER
I20260811 02:51:57.565303 14993 sys_catalog.cc:458] T 00000000000000000000000000000000 P 60ef762b8fc246488dfb261f0b1e8420 [sys.catalog]: This master's current role is: LEADER
I20260811 02:51:57.565831 15002 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 02:51:57.568580 15002 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 02:51:57.574398 15002 catalog_manager.cc:1347] Generated new cluster ID: 689c8075f8ec4a1d8e4b06191791b391
I20260811 02:51:57.574470 15002 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 02:51:57.593884 15002 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 02:51:57.594923 15002 catalog_manager.cc:1504] Loading token signing keys...
I20260811 02:51:57.606343 15002 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 60ef762b8fc246488dfb261f0b1e8420: Generated new TSK 0
I20260811 02:51:57.607218 15002 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.12.138.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.138.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.138.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786416707705494-12843-0/krb5kdc/kudu_127.12.138.193.keytab.
Entry for principal kudu/127.12.138.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786416707705494-12843-0/krb5kdc/kudu_127.12.138.193.keytab.
I20260811 02:51:57.643961 12843 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDawEwS/build/debug/bin/kudu
/tmp/dist-test-taskDawEwS/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786416707705494-12843-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786416707705494-12843-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786416707705494-12843-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786416707705494-12843-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.138.193:0
--local_ip_for_outbound_sockets=127.12.138.193
--webserver_interface=127.12.138.193
--webserver_port=0
--tserver_master_addrs=127.12.138.254:41515
--keytab_file=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786416707705494-12843-0/krb5kdc/kudu_127.12.138.193.keytab
--principal=kudu/127.12.138.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.138.212:41525
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786416707705494-12843-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786416707705494-12843-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786416707705494-12843-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 02:51:57.760998 15015 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 02:51:57.761353 15015 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 02:51:57.761409 15015 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 02:51:57.765645 15015 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 02:51:57.765715 15015 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 02:51:57.765789 15015 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260811 02:51:57.765821 15015 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260811 02:51:57.765898 15015 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.138.193
I20260811 02:51:57.771493 15015 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.138.212:41525
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786416707705494-12843-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786416707705494-12843-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786416707705494-12843-0/krb5kdc/kudu_127.12.138.193.keytab
--principal=kudu/127.12.138.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.138.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786416707705494-12843-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.12.138.193
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.12.138.254:41515
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15015
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.138.193
--log_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786416707705494-12843-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 02:09:49 UTC on ccycloud.quasar-fsnqqc.root.comops.site
build id 4579570
I20260811 02:51:57.772567 15015 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 02:51:57.773605 15015 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 02:51:57.784121 15024 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:51:57.784196 15020 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:51:57.784199 15015 server_base.cc:1034] running on GCE node
W20260811 02:51:57.784420 15021 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:51:57.785008 15015 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 02:51:57.785739 15015 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 02:51:57.786933 15015 hybrid_clock.cc:648] HybridClock initialized: now 1786416717786897 us; error 42 us; skew 500 ppm
Aug 11 02:51:57 dist-test-slave-9gcw krb5kdc[14879](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416717, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.138.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 02:51:57.790084 15015 init.cc:377] Logged in from keytab as kudu/127.12.138.193@KRBTEST.COM (short username kudu)
I20260811 02:51:57.791206 15015 webserver.cc:460] Webserver started at http://127.12.138.193:37669/ using document root <none> and password file <none>
I20260811 02:51:57.791844 15015 fs_manager.cc:359] Metadata directory not provided
I20260811 02:51:57.791914 15015 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 02:51:57.792181 15015 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 02:51:57.794539 15015 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786416707705494-12843-0/minicluster-data/ts-0/data/instance:
uuid: "c8aa4e1129d146d4a47f70f248771868"
format_stamp: "Formatted at 2026-08-11 02:51:57 on dist-test-slave-9gcw"
server_key: "48f72761d638282db43965b5ac42c1a8"
server_key_iv: "cddc55bf43a36f517eb2075ce63d504f"
server_key_version: "encryptionkey@0"
I20260811 02:51:57.795204 15015 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786416707705494-12843-0/minicluster-data/ts-0/wal/instance:
uuid: "c8aa4e1129d146d4a47f70f248771868"
format_stamp: "Formatted at 2026-08-11 02:51:57 on dist-test-slave-9gcw"
server_key: "48f72761d638282db43965b5ac42c1a8"
server_key_iv: "cddc55bf43a36f517eb2075ce63d504f"
server_key_version: "encryptionkey@0"
I20260811 02:51:57.799489 15015 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260811 02:51:57.802306 15033 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 02:51:57.803762 15015 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 02:51:57.803941 15015 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786416707705494-12843-0/minicluster-data/ts-0/data,/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786416707705494-12843-0/minicluster-data/ts-0/wal
uuid: "c8aa4e1129d146d4a47f70f248771868"
format_stamp: "Formatted at 2026-08-11 02:51:57 on dist-test-slave-9gcw"
server_key: "48f72761d638282db43965b5ac42c1a8"
server_key_iv: "cddc55bf43a36f517eb2075ce63d504f"
server_key_version: "encryptionkey@0"
I20260811 02:51:57.804075 15015 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786416707705494-12843-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786416707705494-12843-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786416707705494-12843-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 02:51:57.829049 15015 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 02:51:57.832549 15015 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 02:51:57.832792 15015 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 02:51:57.834641 15015 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 02:51:57.834738 15015 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 02:51:57.834823 15015 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 02:51:57.834872 15015 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 11 02:51:57 dist-test-slave-9gcw krb5kdc[14879](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416717, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.138.193@KRBTEST.COM for kudu/127.12.138.254@KRBTEST.COM
I20260811 02:51:57.846907 15015 rpc_server.cc:307] RPC server started. Bound to: 127.12.138.193:38359
I20260811 02:51:57.846979 15167 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.138.193:38359 every 8 connection(s)
I20260811 02:51:57.847941 15015 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786416707705494-12843-0/minicluster-data/ts-0/data/info.pb
I20260811 02:51:57.850461 14989 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:51:57.835555 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.138.193:34199 (local address 127.12.138.254:41515)
0811 02:51:57.835719 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0811 02:51:57.835724 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 02:51:57.836621 (+ 897us) server_negotiation.cc:407] Connection header received
0811 02:51:57.837707 (+ 1086us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:51:57.837711 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:51:57.837769 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:51:57.837857 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:51:57.839880 (+ 2023us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:51:57.840792 (+ 912us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:51:57.843011 (+ 2219us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:51:57.843390 (+ 379us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:51:57.843518 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:51:57.846287 (+ 2769us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:51:57.846290 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:51:57.846293 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:51:57.846320 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:51:57.848023 (+ 1703us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:51:57.848685 (+ 662us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:51:57.848689 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:51:57.848691 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:51:57.848803 (+ 112us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:51:57.849240 (+ 437us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:51:57.849246 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:51:57.849253 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:51:57.849465 (+ 212us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:51:57.849583 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0811 02:51:57.850127 (+ 544us) server_negotiation.cc:299] Negotiation successful
0811 02:51:57.850277 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260811 02:51:57.851387 15046 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:51:57.835971 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.138.254:41515 (local address 127.12.138.193:34199)
0811 02:51:57.836429 (+ 458us) negotiation.cc:107] Waiting for socket to connect
0811 02:51:57.836480 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0811 02:51:57.837479 (+ 999us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:51:57.838227 (+ 748us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:51:57.838242 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:51:57.838854 (+ 612us) client_negotiation.cc:189] Negotiated authn=SASL
0811 02:51:57.839662 (+ 808us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:51:57.839692 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:51:57.840974 (+ 1282us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:51:57.840982 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:51:57.842846 (+ 1864us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:51:57.842865 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:51:57.843565 (+ 700us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:51:57.843569 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:51:57.843762 (+ 193us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:51:57.844435 (+ 673us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 02:51:57.844462 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0811 02:51:57.846078 (+ 1616us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 02:51:57.848162 (+ 2084us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:51:57.848168 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:51:57.848182 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:51:57.848523 (+ 341us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:51:57.848920 (+ 397us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:51:57.848923 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:51:57.848925 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:51:57.849117 (+ 192us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:51:57.849583 (+ 466us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 02:51:57.849591 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 02:51:57.849862 (+ 271us) client_negotiation.cc:769] Sending connection context
0811 02:51:57.850061 (+ 199us) client_negotiation.cc:240] Negotiation successful
0811 02:51:57.850420 (+ 359us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":254,"thread_start_us":128,"threads_started":1}
I20260811 02:51:57.851980 12843 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDawEwS/build/debug/bin/kudu as pid 15015
I20260811 02:51:57.852133 12843 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786416707705494-12843-0/minicluster-data/ts-0/wal/instance
I20260811 02:51:57.852486 12843 external_mini_cluster.cc:1428] Setting key 62dd0d4bfc1202079e134f9f8668eb82
I20260811 02:51:57.853025 15169 heartbeater.cc:347] Connected to a master server at 127.12.138.254:41515
I20260811 02:51:57.853267 15169 heartbeater.cc:464] Registering TS with master...
I20260811 02:51:57.853803 15169 heartbeater.cc:511] Master 127.12.138.254:41515 requested a full tablet report, sending...
I20260811 02:51:57.855422 14933 ts_manager.cc:194] Registered new tserver with Master: c8aa4e1129d146d4a47f70f248771868 (127.12.138.193:38359)
I20260811 02:51:57.857007 14933 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.12.138.193@KRBTEST.COM'} at 127.12.138.193:34199
WARNING: no policy specified for kudu/127.12.138.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.138.194@KRBTEST.COM" created.
I20260811 02:51:57.868785 14989 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:51:57.855769 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.138.193:44699 (local address 127.12.138.254:41515)
0811 02:51:57.855918 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0811 02:51:57.855926 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 02:51:57.856066 (+ 140us) server_negotiation.cc:407] Connection header received
0811 02:51:57.856222 (+ 156us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:51:57.856226 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:51:57.856273 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:51:57.856325 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:51:57.856965 (+ 640us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:51:57.857873 (+ 908us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:51:57.858764 (+ 891us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:51:57.859083 (+ 319us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:51:57.859200 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:51:57.864724 (+ 5524us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:51:57.864727 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:51:57.864730 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:51:57.864754 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:51:57.866438 (+ 1684us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:51:57.866929 (+ 491us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:51:57.866932 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:51:57.866933 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:51:57.867077 (+ 144us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:51:57.867504 (+ 427us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:51:57.867507 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:51:57.867509 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:51:57.867929 (+ 420us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:51:57.868034 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0811 02:51:57.868412 (+ 378us) server_negotiation.cc:299] Negotiation successful
0811 02:51:57.868566 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260811 02:51:57.869589 15176 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:51:57.855602 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.138.254:41515 (local address 127.12.138.193:44699)
0811 02:51:57.855969 (+ 367us) negotiation.cc:107] Waiting for socket to connect
0811 02:51:57.855985 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 02:51:57.856084 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:51:57.856459 (+ 375us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:51:57.856462 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:51:57.856710 (+ 248us) client_negotiation.cc:189] Negotiated authn=SASL
0811 02:51:57.856827 (+ 117us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:51:57.856833 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:51:57.858045 (+ 1212us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:51:57.858048 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:51:57.858643 (+ 595us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:51:57.858650 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:51:57.859232 (+ 582us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:51:57.859239 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:51:57.859371 (+ 132us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:51:57.864022 (+ 4651us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 02:51:57.864044 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0811 02:51:57.864565 (+ 521us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 02:51:57.866553 (+ 1988us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:51:57.866556 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:51:57.866558 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:51:57.866825 (+ 267us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:51:57.867257 (+ 432us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:51:57.867262 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:51:57.867264 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:51:57.867388 (+ 124us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:51:57.868086 (+ 698us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 02:51:57.868089 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 02:51:57.868206 (+ 117us) client_negotiation.cc:769] Sending connection context
0811 02:51:57.869048 (+ 842us) client_negotiation.cc:240] Negotiation successful
0811 02:51:57.869292 (+ 244us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":281,"thread_start_us":118,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.138.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786416707705494-12843-0/krb5kdc/kudu_127.12.138.194.keytab.
Entry for principal kudu/127.12.138.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786416707705494-12843-0/krb5kdc/kudu_127.12.138.194.keytab.
I20260811 02:51:57.887753 12843 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDawEwS/build/debug/bin/kudu
/tmp/dist-test-taskDawEwS/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786416707705494-12843-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786416707705494-12843-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786416707705494-12843-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786416707705494-12843-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.138.194:0
--local_ip_for_outbound_sockets=127.12.138.194
--webserver_interface=127.12.138.194
--webserver_port=0
--tserver_master_addrs=127.12.138.254:41515
--keytab_file=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786416707705494-12843-0/krb5kdc/kudu_127.12.138.194.keytab
--principal=kudu/127.12.138.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.138.212:41525
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786416707705494-12843-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786416707705494-12843-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786416707705494-12843-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 02:51:58.016928 15179 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 02:51:58.017326 15179 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 02:51:58.017443 15179 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 02:51:58.021831 15179 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 02:51:58.021916 15179 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 02:51:58.022006 15179 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260811 02:51:58.022066 15179 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260811 02:51:58.022127 15179 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.138.194
I20260811 02:51:58.032151 15179 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.138.212:41525
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786416707705494-12843-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786416707705494-12843-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786416707705494-12843-0/krb5kdc/kudu_127.12.138.194.keytab
--principal=kudu/127.12.138.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.138.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786416707705494-12843-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.12.138.194
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.12.138.254:41515
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15179
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.138.194
--log_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786416707705494-12843-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 02:09:49 UTC on ccycloud.quasar-fsnqqc.root.comops.site
build id 4579570
I20260811 02:51:58.033739 15179 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 02:51:58.035238 15179 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 02:51:58.049043 15189 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:51:58.049189 15186 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:51:58.049021 15187 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:51:58.049947 15179 server_base.cc:1034] running on GCE node
I20260811 02:51:58.050555 15179 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 02:51:58.051474 15179 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 02:51:58.053054 15179 hybrid_clock.cc:648] HybridClock initialized: now 1786416718053034 us; error 37 us; skew 500 ppm
Aug 11 02:51:58 dist-test-slave-9gcw krb5kdc[14879](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416718, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.138.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 02:51:58.056785 15179 init.cc:377] Logged in from keytab as kudu/127.12.138.194@KRBTEST.COM (short username kudu)
I20260811 02:51:58.058125 15179 webserver.cc:460] Webserver started at http://127.12.138.194:43861/ using document root <none> and password file <none>
I20260811 02:51:58.058734 15179 fs_manager.cc:359] Metadata directory not provided
I20260811 02:51:58.058841 15179 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 02:51:58.059087 15179 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 02:51:58.061414 15179 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786416707705494-12843-0/minicluster-data/ts-1/data/instance:
uuid: "e6cbb5d02dcb40edb34ab5d41193496c"
format_stamp: "Formatted at 2026-08-11 02:51:58 on dist-test-slave-9gcw"
server_key: "01645678678d5f5e56d547101474c8c7"
server_key_iv: "e08f35c57fb64993cbfcc3fbe810451d"
server_key_version: "encryptionkey@0"
I20260811 02:51:58.062211 15179 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786416707705494-12843-0/minicluster-data/ts-1/wal/instance:
uuid: "e6cbb5d02dcb40edb34ab5d41193496c"
format_stamp: "Formatted at 2026-08-11 02:51:58 on dist-test-slave-9gcw"
server_key: "01645678678d5f5e56d547101474c8c7"
server_key_iv: "e08f35c57fb64993cbfcc3fbe810451d"
server_key_version: "encryptionkey@0"
I20260811 02:51:58.068004 15179 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260811 02:51:58.071012 15196 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 02:51:58.072781 15179 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260811 02:51:58.072908 15179 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786416707705494-12843-0/minicluster-data/ts-1/data,/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786416707705494-12843-0/minicluster-data/ts-1/wal
uuid: "e6cbb5d02dcb40edb34ab5d41193496c"
format_stamp: "Formatted at 2026-08-11 02:51:58 on dist-test-slave-9gcw"
server_key: "01645678678d5f5e56d547101474c8c7"
server_key_iv: "e08f35c57fb64993cbfcc3fbe810451d"
server_key_version: "encryptionkey@0"
I20260811 02:51:58.073052 15179 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786416707705494-12843-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786416707705494-12843-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786416707705494-12843-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 02:51:58.110363 15179 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 02:51:58.114078 15179 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 02:51:58.114344 15179 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 02:51:58.116030 15179 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 02:51:58.116118 15179 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 02:51:58.116214 15179 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 02:51:58.116256 15179 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 11 02:51:58 dist-test-slave-9gcw krb5kdc[14879](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416718, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.138.194@KRBTEST.COM for kudu/127.12.138.254@KRBTEST.COM
I20260811 02:51:58.128692 15179 rpc_server.cc:307] RPC server started. Bound to: 127.12.138.194:33581
I20260811 02:51:58.129616 15331 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.138.194:33581 every 8 connection(s)
I20260811 02:51:58.129942 15179 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786416707705494-12843-0/minicluster-data/ts-1/data/info.pb
I20260811 02:51:58.131541 14989 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:51:58.116821 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.138.194:54803 (local address 127.12.138.254:41515)
0811 02:51:58.116958 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0811 02:51:58.116963 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 02:51:58.118135 (+ 1172us) server_negotiation.cc:407] Connection header received
0811 02:51:58.119558 (+ 1423us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:51:58.119562 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:51:58.119627 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:51:58.119711 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:51:58.121287 (+ 1576us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:51:58.121852 (+ 565us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:51:58.123312 (+ 1460us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:51:58.123553 (+ 241us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:51:58.123673 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:51:58.127071 (+ 3398us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:51:58.127079 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:51:58.127086 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:51:58.127125 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:51:58.129119 (+ 1994us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:51:58.129798 (+ 679us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:51:58.129802 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:51:58.129804 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:51:58.129912 (+ 108us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:51:58.130317 (+ 405us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:51:58.130320 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:51:58.130322 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:51:58.130560 (+ 238us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:51:58.130673 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0811 02:51:58.131293 (+ 620us) server_negotiation.cc:299] Negotiation successful
0811 02:51:58.131413 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260811 02:51:58.132292 15205 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:51:58.117393 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.138.254:41515 (local address 127.12.138.194:54803)
0811 02:51:58.117942 (+ 549us) negotiation.cc:107] Waiting for socket to connect
0811 02:51:58.118024 (+ 82us) client_negotiation.cc:174] Beginning negotiation
0811 02:51:58.119302 (+ 1278us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:51:58.119898 (+ 596us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:51:58.119911 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:51:58.120448 (+ 537us) client_negotiation.cc:189] Negotiated authn=SASL
0811 02:51:58.121107 (+ 659us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:51:58.121131 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:51:58.121989 (+ 858us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:51:58.121993 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:51:58.123176 (+ 1183us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:51:58.123186 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:51:58.123728 (+ 542us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:51:58.123732 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:51:58.123934 (+ 202us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:51:58.124646 (+ 712us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 02:51:58.124665 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0811 02:51:58.126878 (+ 2213us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 02:51:58.129255 (+ 2377us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:51:58.129260 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:51:58.129273 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:51:58.129666 (+ 393us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:51:58.130021 (+ 355us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:51:58.130024 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:51:58.130027 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:51:58.130208 (+ 181us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:51:58.130729 (+ 521us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 02:51:58.130735 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 02:51:58.131040 (+ 305us) client_negotiation.cc:769] Sending connection context
0811 02:51:58.131196 (+ 156us) client_negotiation.cc:240] Negotiation successful
0811 02:51:58.131448 (+ 252us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":260,"thread_start_us":132,"threads_started":1}
I20260811 02:51:58.133806 15332 heartbeater.cc:347] Connected to a master server at 127.12.138.254:41515
I20260811 02:51:58.133988 15332 heartbeater.cc:464] Registering TS with master...
I20260811 02:51:58.134524 15332 heartbeater.cc:511] Master 127.12.138.254:41515 requested a full tablet report, sending...
I20260811 02:51:58.135622 14928 ts_manager.cc:194] Registered new tserver with Master: e6cbb5d02dcb40edb34ab5d41193496c (127.12.138.194:33581)
I20260811 02:51:58.136406 14928 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.12.138.194@KRBTEST.COM'} at 127.12.138.194:54803
I20260811 02:51:58.136818 12843 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDawEwS/build/debug/bin/kudu as pid 15179
I20260811 02:51:58.136966 12843 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786416707705494-12843-0/minicluster-data/ts-1/wal/instance
I20260811 02:51:58.137363 12843 external_mini_cluster.cc:1428] Setting key 2b4e7c524da775747cff6d3a3e5ee2ed
I20260811 02:51:58.144989 15338 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:51:58.136098 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.138.254:41515 (local address 127.12.138.194:37775)
0811 02:51:58.136368 (+ 270us) negotiation.cc:107] Waiting for socket to connect
0811 02:51:58.136381 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 02:51:58.136463 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:51:58.137287 (+ 824us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:51:58.137291 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:51:58.137532 (+ 241us) client_negotiation.cc:189] Negotiated authn=SASL
0811 02:51:58.137669 (+ 137us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:51:58.137677 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:51:58.138689 (+ 1012us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:51:58.138691 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:51:58.139555 (+ 864us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:51:58.139570 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:51:58.140167 (+ 597us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:51:58.140173 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:51:58.140302 (+ 129us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:51:58.140834 (+ 532us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 02:51:58.140854 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0811 02:51:58.141312 (+ 458us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 02:51:58.143290 (+ 1978us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:51:58.143293 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:51:58.143295 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:51:58.143518 (+ 223us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:51:58.143840 (+ 322us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:51:58.143842 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:51:58.143844 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:51:58.143930 (+ 86us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:51:58.144515 (+ 585us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 02:51:58.144517 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 02:51:58.144599 (+ 82us) client_negotiation.cc:769] Sending connection context
0811 02:51:58.144711 (+ 112us) client_negotiation.cc:240] Negotiation successful
0811 02:51:58.144839 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":174,"thread_start_us":94,"threads_started":1}
I20260811 02:51:58.145116 14989 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:51:58.136188 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.138.194:37775 (local address 127.12.138.254:41515)
0811 02:51:58.136303 (+ 115us) server_negotiation.cc:206] Beginning negotiation
0811 02:51:58.136307 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 02:51:58.136458 (+ 151us) server_negotiation.cc:407] Connection header received
0811 02:51:58.136577 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:51:58.136581 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:51:58.136641 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:51:58.136708 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:51:58.137826 (+ 1118us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:51:58.138444 (+ 618us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:51:58.139797 (+ 1353us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:51:58.140039 (+ 242us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:51:58.140134 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:51:58.141450 (+ 1316us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:51:58.141453 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:51:58.141456 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:51:58.141479 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:51:58.143165 (+ 1686us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:51:58.143627 (+ 462us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:51:58.143633 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:51:58.143638 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:51:58.143734 (+ 96us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:51:58.144028 (+ 294us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:51:58.144034 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:51:58.144039 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:51:58.144407 (+ 368us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:51:58.144522 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0811 02:51:58.144788 (+ 266us) server_negotiation.cc:299] Negotiation successful
0811 02:51:58.144933 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
WARNING: no policy specified for kudu/127.12.138.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.138.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.138.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786416707705494-12843-0/krb5kdc/kudu_127.12.138.195.keytab.
Entry for principal kudu/127.12.138.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786416707705494-12843-0/krb5kdc/kudu_127.12.138.195.keytab.
I20260811 02:51:58.168597 12843 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDawEwS/build/debug/bin/kudu
/tmp/dist-test-taskDawEwS/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786416707705494-12843-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786416707705494-12843-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786416707705494-12843-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786416707705494-12843-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.138.195:0
--local_ip_for_outbound_sockets=127.12.138.195
--webserver_interface=127.12.138.195
--webserver_port=0
--tserver_master_addrs=127.12.138.254:41515
--keytab_file=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786416707705494-12843-0/krb5kdc/kudu_127.12.138.195.keytab
--principal=kudu/127.12.138.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.138.212:41525
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786416707705494-12843-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786416707705494-12843-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786416707705494-12843-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 02:51:58.288486 15341 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 02:51:58.288800 15341 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 02:51:58.288883 15341 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 02:51:58.292912 15341 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 02:51:58.293013 15341 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 02:51:58.293087 15341 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260811 02:51:58.293123 15341 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260811 02:51:58.293205 15341 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.138.195
I20260811 02:51:58.298812 15341 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.138.212:41525
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786416707705494-12843-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786416707705494-12843-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786416707705494-12843-0/krb5kdc/kudu_127.12.138.195.keytab
--principal=kudu/127.12.138.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.138.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786416707705494-12843-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.12.138.195
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.12.138.254:41515
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15341
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.138.195
--log_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786416707705494-12843-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 02:09:49 UTC on ccycloud.quasar-fsnqqc.root.comops.site
build id 4579570
I20260811 02:51:58.300041 15341 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 02:51:58.301100 15341 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 02:51:58.311666 15347 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:51:58.311674 15350 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:51:58.311674 15348 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:51:58.311937 15341 server_base.cc:1034] running on GCE node
I20260811 02:51:58.312494 15341 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 02:51:58.313290 15341 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 02:51:58.314497 15341 hybrid_clock.cc:648] HybridClock initialized: now 1786416718314464 us; error 47 us; skew 500 ppm
Aug 11 02:51:58 dist-test-slave-9gcw krb5kdc[14879](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416718, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.138.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 02:51:58.317641 15341 init.cc:377] Logged in from keytab as kudu/127.12.138.195@KRBTEST.COM (short username kudu)
I20260811 02:51:58.318602 15341 webserver.cc:460] Webserver started at http://127.12.138.195:35011/ using document root <none> and password file <none>
I20260811 02:51:58.319202 15341 fs_manager.cc:359] Metadata directory not provided
I20260811 02:51:58.319299 15341 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 02:51:58.319557 15341 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 02:51:58.321798 15341 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786416707705494-12843-0/minicluster-data/ts-2/data/instance:
uuid: "c0f254aed5f64d8b8f923c83b2b067b7"
format_stamp: "Formatted at 2026-08-11 02:51:58 on dist-test-slave-9gcw"
server_key: "5460ed8ea3ef17d3e684b503e0e6004c"
server_key_iv: "5bff189f04f6d5ee45803fa00900c3b6"
server_key_version: "encryptionkey@0"
I20260811 02:51:58.322418 15341 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786416707705494-12843-0/minicluster-data/ts-2/wal/instance:
uuid: "c0f254aed5f64d8b8f923c83b2b067b7"
format_stamp: "Formatted at 2026-08-11 02:51:58 on dist-test-slave-9gcw"
server_key: "5460ed8ea3ef17d3e684b503e0e6004c"
server_key_iv: "5bff189f04f6d5ee45803fa00900c3b6"
server_key_version: "encryptionkey@0"
I20260811 02:51:58.326507 15341 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.000s sys 0.004s
I20260811 02:51:58.329071 15360 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 02:51:58.330590 15341 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260811 02:51:58.330732 15341 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786416707705494-12843-0/minicluster-data/ts-2/data,/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786416707705494-12843-0/minicluster-data/ts-2/wal
uuid: "c0f254aed5f64d8b8f923c83b2b067b7"
format_stamp: "Formatted at 2026-08-11 02:51:58 on dist-test-slave-9gcw"
server_key: "5460ed8ea3ef17d3e684b503e0e6004c"
server_key_iv: "5bff189f04f6d5ee45803fa00900c3b6"
server_key_version: "encryptionkey@0"
I20260811 02:51:58.330850 15341 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786416707705494-12843-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786416707705494-12843-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786416707705494-12843-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 02:51:58.345884 15341 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 02:51:58.349426 15341 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 02:51:58.349659 15341 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 02:51:58.351259 15341 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 02:51:58.351349 15341 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 02:51:58.351444 15341 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 02:51:58.351496 15341 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 11 02:51:58 dist-test-slave-9gcw krb5kdc[14879](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416718, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.138.195@KRBTEST.COM for kudu/127.12.138.254@KRBTEST.COM
I20260811 02:51:58.362933 15341 rpc_server.cc:307] RPC server started. Bound to: 127.12.138.195:42421
I20260811 02:51:58.363336 15498 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.138.195:42421 every 8 connection(s)
I20260811 02:51:58.364017 15341 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786416707705494-12843-0/minicluster-data/ts-2/data/info.pb
I20260811 02:51:58.366230 12843 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDawEwS/build/debug/bin/kudu as pid 15341
I20260811 02:51:58.366362 12843 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786416707705494-12843-0/minicluster-data/ts-2/wal/instance
I20260811 02:51:58.366428 14989 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:51:58.352360 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.138.195:50711 (local address 127.12.138.254:41515)
0811 02:51:58.352491 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0811 02:51:58.352495 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 02:51:58.353578 (+ 1083us) server_negotiation.cc:407] Connection header received
0811 02:51:58.354955 (+ 1377us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:51:58.354959 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:51:58.355022 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:51:58.355123 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:51:58.356911 (+ 1788us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:51:58.357822 (+ 911us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:51:58.359123 (+ 1301us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:51:58.359407 (+ 284us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:51:58.359527 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:51:58.362460 (+ 2933us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:51:58.362463 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:51:58.362466 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:51:58.362493 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:51:58.364099 (+ 1606us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:51:58.364707 (+ 608us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:51:58.364711 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:51:58.364715 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:51:58.364814 (+ 99us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:51:58.365242 (+ 428us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:51:58.365245 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:51:58.365247 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:51:58.365496 (+ 249us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:51:58.365619 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0811 02:51:58.366092 (+ 473us) server_negotiation.cc:299] Negotiation successful
0811 02:51:58.366229 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260811 02:51:58.366727 12843 external_mini_cluster.cc:1428] Setting key 7e4ac7a489c53df9ccae9f29cacc2a66
I20260811 02:51:58.367218 15371 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:51:58.352888 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.138.254:41515 (local address 127.12.138.195:50711)
0811 02:51:58.353427 (+ 539us) negotiation.cc:107] Waiting for socket to connect
0811 02:51:58.353470 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0811 02:51:58.354571 (+ 1101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:51:58.355351 (+ 780us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:51:58.355363 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:51:58.355932 (+ 569us) client_negotiation.cc:189] Negotiated authn=SASL
0811 02:51:58.356707 (+ 775us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:51:58.356732 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:51:58.357993 (+ 1261us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:51:58.357997 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:51:58.358985 (+ 988us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:51:58.358995 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:51:58.359569 (+ 574us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:51:58.359573 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:51:58.359796 (+ 223us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:51:58.360441 (+ 645us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 02:51:58.360458 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 02:51:58.362290 (+ 1832us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 02:51:58.364223 (+ 1933us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:51:58.364228 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:51:58.364245 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:51:58.364562 (+ 317us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:51:58.364906 (+ 344us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:51:58.364911 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:51:58.364914 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:51:58.365091 (+ 177us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:51:58.365599 (+ 508us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 02:51:58.365603 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 02:51:58.365831 (+ 228us) client_negotiation.cc:769] Sending connection context
0811 02:51:58.365988 (+ 157us) client_negotiation.cc:240] Negotiation successful
0811 02:51:58.366239 (+ 251us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":342,"thread_start_us":155,"threads_started":1}
I20260811 02:51:58.368583 15499 heartbeater.cc:347] Connected to a master server at 127.12.138.254:41515
I20260811 02:51:58.368731 15499 heartbeater.cc:464] Registering TS with master...
I20260811 02:51:58.369345 15499 heartbeater.cc:511] Master 127.12.138.254:41515 requested a full tablet report, sending...
I20260811 02:51:58.370483 14928 ts_manager.cc:194] Registered new tserver with Master: c0f254aed5f64d8b8f923c83b2b067b7 (127.12.138.195:42421)
I20260811 02:51:58.371210 14928 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.12.138.195@KRBTEST.COM'} at 127.12.138.195:50711
I20260811 02:51:58.378144 14989 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:51:58.371070 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.138.195:39805 (local address 127.12.138.254:41515)
0811 02:51:58.371177 (+ 107us) server_negotiation.cc:206] Beginning negotiation
0811 02:51:58.371181 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 02:51:58.371382 (+ 201us) server_negotiation.cc:407] Connection header received
0811 02:51:58.371510 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:51:58.371513 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:51:58.371560 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:51:58.371650 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:51:58.372088 (+ 438us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:51:58.372599 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:51:58.373614 (+ 1015us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:51:58.373844 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:51:58.373957 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:51:58.375070 (+ 1113us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:51:58.375074 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:51:58.375098 (+ 24us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:51:58.375127 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:51:58.376422 (+ 1295us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:51:58.376856 (+ 434us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:51:58.376859 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:51:58.376861 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:51:58.376944 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:51:58.377257 (+ 313us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:51:58.377260 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:51:58.377262 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:51:58.377447 (+ 185us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:51:58.377559 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0811 02:51:58.377883 (+ 324us) server_negotiation.cc:299] Negotiation successful
0811 02:51:58.377998 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260811 02:51:58.378158 15507 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:51:58.371007 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.138.254:41515 (local address 127.12.138.195:39805)
0811 02:51:58.371260 (+ 253us) negotiation.cc:107] Waiting for socket to connect
0811 02:51:58.371276 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 02:51:58.371401 (+ 125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:51:58.371679 (+ 278us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:51:58.371681 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:51:58.371849 (+ 168us) client_negotiation.cc:189] Negotiated authn=SASL
0811 02:51:58.371972 (+ 123us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:51:58.371977 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:51:58.372730 (+ 753us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:51:58.372733 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:51:58.373492 (+ 759us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:51:58.373502 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:51:58.373960 (+ 458us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:51:58.373964 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:51:58.374055 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:51:58.374588 (+ 533us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 02:51:58.374601 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0811 02:51:58.374932 (+ 331us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 02:51:58.376538 (+ 1606us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:51:58.376542 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:51:58.376545 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:51:58.376745 (+ 200us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:51:58.377039 (+ 294us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:51:58.377043 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:51:58.377045 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:51:58.377131 (+ 86us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:51:58.377550 (+ 419us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 02:51:58.377553 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 02:51:58.377656 (+ 103us) client_negotiation.cc:769] Sending connection context
0811 02:51:58.377810 (+ 154us) client_negotiation.cc:240] Negotiation successful
0811 02:51:58.377965 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":173,"mutex_wait_us":3,"thread_start_us":96,"threads_started":1}
I20260811 02:51:58.380918 12843 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 11 02:51:58 dist-test-slave-9gcw krb5kdc[14879](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416718, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 11 02:51:58 dist-test-slave-9gcw krb5kdc[14879](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416718, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.12.138.254@KRBTEST.COM
I20260811 02:51:58.409281 14989 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:51:58.400562 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50218 (local address 127.12.138.254:41515)
0811 02:51:58.400710 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0811 02:51:58.400714 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 02:51:58.400727 (+ 13us) server_negotiation.cc:407] Connection header received
0811 02:51:58.400778 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:51:58.400780 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:51:58.400828 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:51:58.400910 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:51:58.401801 (+ 891us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:51:58.402341 (+ 540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:51:58.403223 (+ 882us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:51:58.403429 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:51:58.403505 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:51:58.406155 (+ 2650us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:51:58.406158 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:51:58.406161 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:51:58.406186 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:51:58.407616 (+ 1430us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:51:58.408063 (+ 447us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:51:58.408066 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:51:58.408067 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:51:58.408143 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:51:58.408401 (+ 258us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:51:58.408404 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:51:58.408406 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:51:58.408593 (+ 187us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:51:58.408694 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0811 02:51:58.408983 (+ 289us) server_negotiation.cc:299] Negotiation successful
0811 02:51:58.409088 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260811 02:51:58.412407 14928 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50218:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 02:51:58.415666 14928 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 02:51:58.430634 15518 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:51:58.424794 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.138.194:33581 (local address 127.0.0.1:53890)
0811 02:51:58.425217 (+ 423us) negotiation.cc:107] Waiting for socket to connect
0811 02:51:58.425273 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0811 02:51:58.425421 (+ 148us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:51:58.426144 (+ 723us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:51:58.426150 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:51:58.426176 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 02:51:58.426343 (+ 167us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:51:58.426350 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:51:58.427760 (+ 1410us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:51:58.427765 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:51:58.428643 (+ 878us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:51:58.428652 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:51:58.430014 (+ 1362us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:51:58.430018 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:51:58.430153 (+ 135us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:51:58.430213 (+ 60us) client_negotiation.cc:769] Sending connection context
0811 02:51:58.430349 (+ 136us) client_negotiation.cc:240] Negotiation successful
0811 02:51:58.430479 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":181,"mutex_wait_us":2,"thread_start_us":100,"threads_started":1}
I20260811 02:51:58.430641 15519 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:51:58.424928 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53890 (local address 127.12.138.194:33581)
0811 02:51:58.425466 (+ 538us) server_negotiation.cc:206] Beginning negotiation
0811 02:51:58.425481 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0811 02:51:58.425509 (+ 28us) server_negotiation.cc:407] Connection header received
0811 02:51:58.425593 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:51:58.425597 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:51:58.425756 (+ 159us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:51:58.425942 (+ 186us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 02:51:58.426472 (+ 530us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:51:58.427641 (+ 1169us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:51:58.428776 (+ 1135us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:51:58.429662 (+ 886us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:51:58.429803 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:51:58.429950 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0811 02:51:58.430386 (+ 436us) server_negotiation.cc:299] Negotiation successful
0811 02:51:58.430484 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":434,"thread_start_us":123,"threads_started":1}
I20260811 02:51:58.431990 15521 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:51:58.425291 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.138.195:42421 (local address 127.0.0.1:40906)
0811 02:51:58.425967 (+ 676us) negotiation.cc:107] Waiting for socket to connect
0811 02:51:58.425982 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 02:51:58.426092 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:51:58.427059 (+ 967us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:51:58.427063 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:51:58.427081 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 02:51:58.427207 (+ 126us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:51:58.427216 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:51:58.428946 (+ 1730us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:51:58.428949 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:51:58.430131 (+ 1182us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:51:58.430140 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:51:58.431607 (+ 1467us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:51:58.431611 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:51:58.431710 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:51:58.431734 (+ 24us) client_negotiation.cc:769] Sending connection context
0811 02:51:58.431798 (+ 64us) client_negotiation.cc:240] Negotiation successful
0811 02:51:58.431860 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":602,"mutex_wait_us":6,"thread_start_us":113,"threads_started":1}
I20260811 02:51:58.432039 15523 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:51:58.426186 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40906 (local address 127.12.138.195:42421)
0811 02:51:58.426536 (+ 350us) server_negotiation.cc:206] Beginning negotiation
0811 02:51:58.426549 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0811 02:51:58.426576 (+ 27us) server_negotiation.cc:407] Connection header received
0811 02:51:58.426661 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:51:58.426668 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:51:58.426894 (+ 226us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:51:58.427372 (+ 478us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 02:51:58.427508 (+ 136us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:51:58.428821 (+ 1313us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:51:58.430264 (+ 1443us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:51:58.431471 (+ 1207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:51:58.431635 (+ 164us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:51:58.431765 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0811 02:51:58.431825 (+ 60us) server_negotiation.cc:299] Negotiation successful
0811 02:51:58.431916 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":238,"thread_start_us":147,"threads_started":1}
I20260811 02:51:58.432634 15520 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:51:58.425711 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.138.193:38359 (local address 127.0.0.1:37042)
0811 02:51:58.426643 (+ 932us) negotiation.cc:107] Waiting for socket to connect
0811 02:51:58.426668 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0811 02:51:58.426766 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:51:58.427508 (+ 742us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:51:58.427511 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:51:58.427529 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 02:51:58.427625 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:51:58.427632 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:51:58.429090 (+ 1458us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:51:58.429095 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:51:58.430377 (+ 1282us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:51:58.430389 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:51:58.432255 (+ 1866us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:51:58.432258 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:51:58.432351 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:51:58.432373 (+ 22us) client_negotiation.cc:769] Sending connection context
0811 02:51:58.432446 (+ 73us) client_negotiation.cc:240] Negotiation successful
0811 02:51:58.432510 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":856,"mutex_wait_us":1,"thread_start_us":116,"threads_started":1}
I20260811 02:51:58.432924 15522 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:51:58.425918 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37042 (local address 127.12.138.193:38359)
0811 02:51:58.426285 (+ 367us) server_negotiation.cc:206] Beginning negotiation
0811 02:51:58.426293 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 02:51:58.426869 (+ 576us) server_negotiation.cc:407] Connection header received
0811 02:51:58.426963 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:51:58.426971 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:51:58.427176 (+ 205us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:51:58.427350 (+ 174us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 02:51:58.427787 (+ 437us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:51:58.428963 (+ 1176us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:51:58.430534 (+ 1571us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:51:58.431471 (+ 937us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:51:58.431622 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:51:58.431765 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0811 02:51:58.432486 (+ 721us) server_negotiation.cc:299] Negotiation successful
0811 02:51:58.432595 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":272,"thread_start_us":70,"threads_started":1}
I20260811 02:51:58.434031 15258 tablet_service.cc:1467] Processing CreateTablet for tablet bd2af0cce5e14cafa4ef26d4142e160d (DEFAULT_TABLE table=test-table [id=a70705a1c7c04407b51153325661c6fc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 02:51:58.435253 15258 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bd2af0cce5e14cafa4ef26d4142e160d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 02:51:58.434993 15421 tablet_service.cc:1467] Processing CreateTablet for tablet bd2af0cce5e14cafa4ef26d4142e160d (DEFAULT_TABLE table=test-table [id=a70705a1c7c04407b51153325661c6fc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 02:51:58.436076 15421 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bd2af0cce5e14cafa4ef26d4142e160d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 02:51:58.436487 15091 tablet_service.cc:1467] Processing CreateTablet for tablet bd2af0cce5e14cafa4ef26d4142e160d (DEFAULT_TABLE table=test-table [id=a70705a1c7c04407b51153325661c6fc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 02:51:58.437563 15091 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bd2af0cce5e14cafa4ef26d4142e160d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 02:51:58.443841 15524 tablet_bootstrap.cc:492] T bd2af0cce5e14cafa4ef26d4142e160d P e6cbb5d02dcb40edb34ab5d41193496c: Bootstrap starting.
I20260811 02:51:58.444918 15526 tablet_bootstrap.cc:492] T bd2af0cce5e14cafa4ef26d4142e160d P c8aa4e1129d146d4a47f70f248771868: Bootstrap starting.
I20260811 02:51:58.445436 15525 tablet_bootstrap.cc:492] T bd2af0cce5e14cafa4ef26d4142e160d P c0f254aed5f64d8b8f923c83b2b067b7: Bootstrap starting.
I20260811 02:51:58.446928 15524 tablet_bootstrap.cc:654] T bd2af0cce5e14cafa4ef26d4142e160d P e6cbb5d02dcb40edb34ab5d41193496c: Neither blocks nor log segments found. Creating new log.
I20260811 02:51:58.447171 15525 tablet_bootstrap.cc:654] T bd2af0cce5e14cafa4ef26d4142e160d P c0f254aed5f64d8b8f923c83b2b067b7: Neither blocks nor log segments found. Creating new log.
I20260811 02:51:58.447222 15526 tablet_bootstrap.cc:654] T bd2af0cce5e14cafa4ef26d4142e160d P c8aa4e1129d146d4a47f70f248771868: Neither blocks nor log segments found. Creating new log.
I20260811 02:51:58.447940 15525 log.cc:824] T bd2af0cce5e14cafa4ef26d4142e160d P c0f254aed5f64d8b8f923c83b2b067b7: Log is configured to *not* fsync() on all Append() calls
I20260811 02:51:58.448089 15526 log.cc:824] T bd2af0cce5e14cafa4ef26d4142e160d P c8aa4e1129d146d4a47f70f248771868: Log is configured to *not* fsync() on all Append() calls
I20260811 02:51:58.448089 15524 log.cc:824] T bd2af0cce5e14cafa4ef26d4142e160d P e6cbb5d02dcb40edb34ab5d41193496c: Log is configured to *not* fsync() on all Append() calls
I20260811 02:51:58.450408 15524 tablet_bootstrap.cc:492] T bd2af0cce5e14cafa4ef26d4142e160d P e6cbb5d02dcb40edb34ab5d41193496c: No bootstrap required, opened a new log
I20260811 02:51:58.450408 15526 tablet_bootstrap.cc:492] T bd2af0cce5e14cafa4ef26d4142e160d P c8aa4e1129d146d4a47f70f248771868: No bootstrap required, opened a new log
I20260811 02:51:58.450634 15524 ts_tablet_manager.cc:1397] T bd2af0cce5e14cafa4ef26d4142e160d P e6cbb5d02dcb40edb34ab5d41193496c: Time spent bootstrapping tablet: real 0.007s user 0.000s sys 0.005s
I20260811 02:51:58.450632 15526 ts_tablet_manager.cc:1397] T bd2af0cce5e14cafa4ef26d4142e160d P c8aa4e1129d146d4a47f70f248771868: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260811 02:51:58.453811 15525 tablet_bootstrap.cc:492] T bd2af0cce5e14cafa4ef26d4142e160d P c0f254aed5f64d8b8f923c83b2b067b7: No bootstrap required, opened a new log
I20260811 02:51:58.454082 15525 ts_tablet_manager.cc:1397] T bd2af0cce5e14cafa4ef26d4142e160d P c0f254aed5f64d8b8f923c83b2b067b7: Time spent bootstrapping tablet: real 0.009s user 0.003s sys 0.001s
I20260811 02:51:58.455448 15524 raft_consensus.cc:348] T bd2af0cce5e14cafa4ef26d4142e160d P e6cbb5d02dcb40edb34ab5d41193496c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6cbb5d02dcb40edb34ab5d41193496c" member_type: VOTER last_known_addr { host: "127.12.138.194" port: 33581 } } peers { permanent_uuid: "c8aa4e1129d146d4a47f70f248771868" member_type: VOTER last_known_addr { host: "127.12.138.193" port: 38359 } } peers { permanent_uuid: "c0f254aed5f64d8b8f923c83b2b067b7" member_type: VOTER last_known_addr { host: "127.12.138.195" port: 42421 } }
I20260811 02:51:58.455772 15524 raft_consensus.cc:374] T bd2af0cce5e14cafa4ef26d4142e160d P e6cbb5d02dcb40edb34ab5d41193496c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 02:51:58.455830 15524 raft_consensus.cc:729] T bd2af0cce5e14cafa4ef26d4142e160d P e6cbb5d02dcb40edb34ab5d41193496c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e6cbb5d02dcb40edb34ab5d41193496c, State: Initialized, Role: FOLLOWER
I20260811 02:51:58.455966 15526 raft_consensus.cc:348] T bd2af0cce5e14cafa4ef26d4142e160d P c8aa4e1129d146d4a47f70f248771868 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6cbb5d02dcb40edb34ab5d41193496c" member_type: VOTER last_known_addr { host: "127.12.138.194" port: 33581 } } peers { permanent_uuid: "c8aa4e1129d146d4a47f70f248771868" member_type: VOTER last_known_addr { host: "127.12.138.193" port: 38359 } } peers { permanent_uuid: "c0f254aed5f64d8b8f923c83b2b067b7" member_type: VOTER last_known_addr { host: "127.12.138.195" port: 42421 } }
I20260811 02:51:58.456236 15526 raft_consensus.cc:374] T bd2af0cce5e14cafa4ef26d4142e160d P c8aa4e1129d146d4a47f70f248771868 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 02:51:58.456308 15526 raft_consensus.cc:729] T bd2af0cce5e14cafa4ef26d4142e160d P c8aa4e1129d146d4a47f70f248771868 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c8aa4e1129d146d4a47f70f248771868, State: Initialized, Role: FOLLOWER
I20260811 02:51:58.456358 15524 consensus_queue.cc:260] T bd2af0cce5e14cafa4ef26d4142e160d P e6cbb5d02dcb40edb34ab5d41193496c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6cbb5d02dcb40edb34ab5d41193496c" member_type: VOTER last_known_addr { host: "127.12.138.194" port: 33581 } } peers { permanent_uuid: "c8aa4e1129d146d4a47f70f248771868" member_type: VOTER last_known_addr { host: "127.12.138.193" port: 38359 } } peers { permanent_uuid: "c0f254aed5f64d8b8f923c83b2b067b7" member_type: VOTER last_known_addr { host: "127.12.138.195" port: 42421 } }
I20260811 02:51:58.456843 15526 consensus_queue.cc:260] T bd2af0cce5e14cafa4ef26d4142e160d P c8aa4e1129d146d4a47f70f248771868 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6cbb5d02dcb40edb34ab5d41193496c" member_type: VOTER last_known_addr { host: "127.12.138.194" port: 33581 } } peers { permanent_uuid: "c8aa4e1129d146d4a47f70f248771868" member_type: VOTER last_known_addr { host: "127.12.138.193" port: 38359 } } peers { permanent_uuid: "c0f254aed5f64d8b8f923c83b2b067b7" member_type: VOTER last_known_addr { host: "127.12.138.195" port: 42421 } }
I20260811 02:51:58.457293 15332 heartbeater.cc:503] Master 127.12.138.254:41515 was elected leader, sending a full tablet report...
I20260811 02:51:58.457921 15526 ts_tablet_manager.cc:1428] T bd2af0cce5e14cafa4ef26d4142e160d P c8aa4e1129d146d4a47f70f248771868: Time spent starting tablet: real 0.007s user 0.004s sys 0.004s
I20260811 02:51:58.458285 15169 heartbeater.cc:503] Master 127.12.138.254:41515 was elected leader, sending a full tablet report...
I20260811 02:51:58.458235 15525 raft_consensus.cc:348] T bd2af0cce5e14cafa4ef26d4142e160d P c0f254aed5f64d8b8f923c83b2b067b7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6cbb5d02dcb40edb34ab5d41193496c" member_type: VOTER last_known_addr { host: "127.12.138.194" port: 33581 } } peers { permanent_uuid: "c8aa4e1129d146d4a47f70f248771868" member_type: VOTER last_known_addr { host: "127.12.138.193" port: 38359 } } peers { permanent_uuid: "c0f254aed5f64d8b8f923c83b2b067b7" member_type: VOTER last_known_addr { host: "127.12.138.195" port: 42421 } }
I20260811 02:51:58.458525 15525 raft_consensus.cc:374] T bd2af0cce5e14cafa4ef26d4142e160d P c0f254aed5f64d8b8f923c83b2b067b7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 02:51:58.458588 15525 raft_consensus.cc:729] T bd2af0cce5e14cafa4ef26d4142e160d P c0f254aed5f64d8b8f923c83b2b067b7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c0f254aed5f64d8b8f923c83b2b067b7, State: Initialized, Role: FOLLOWER
I20260811 02:51:58.459018 15525 consensus_queue.cc:260] T bd2af0cce5e14cafa4ef26d4142e160d P c0f254aed5f64d8b8f923c83b2b067b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6cbb5d02dcb40edb34ab5d41193496c" member_type: VOTER last_known_addr { host: "127.12.138.194" port: 33581 } } peers { permanent_uuid: "c8aa4e1129d146d4a47f70f248771868" member_type: VOTER last_known_addr { host: "127.12.138.193" port: 38359 } } peers { permanent_uuid: "c0f254aed5f64d8b8f923c83b2b067b7" member_type: VOTER last_known_addr { host: "127.12.138.195" port: 42421 } }
I20260811 02:51:58.459303 15524 ts_tablet_manager.cc:1428] T bd2af0cce5e14cafa4ef26d4142e160d P e6cbb5d02dcb40edb34ab5d41193496c: Time spent starting tablet: real 0.009s user 0.008s sys 0.000s
I20260811 02:51:58.459879 15499 heartbeater.cc:503] Master 127.12.138.254:41515 was elected leader, sending a full tablet report...
I20260811 02:51:58.460045 15525 ts_tablet_manager.cc:1428] T bd2af0cce5e14cafa4ef26d4142e160d P c0f254aed5f64d8b8f923c83b2b067b7: Time spent starting tablet: real 0.006s user 0.002s sys 0.004s
I20260811 02:51:58.569859 15532 raft_consensus.cc:482] T bd2af0cce5e14cafa4ef26d4142e160d P c8aa4e1129d146d4a47f70f248771868 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 02:51:58.570107 15532 raft_consensus.cc:504] T bd2af0cce5e14cafa4ef26d4142e160d P c8aa4e1129d146d4a47f70f248771868 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6cbb5d02dcb40edb34ab5d41193496c" member_type: VOTER last_known_addr { host: "127.12.138.194" port: 33581 } } peers { permanent_uuid: "c8aa4e1129d146d4a47f70f248771868" member_type: VOTER last_known_addr { host: "127.12.138.193" port: 38359 } } peers { permanent_uuid: "c0f254aed5f64d8b8f923c83b2b067b7" member_type: VOTER last_known_addr { host: "127.12.138.195" port: 42421 } }
I20260811 02:51:58.571503 15532 leader_election.cc:288] T bd2af0cce5e14cafa4ef26d4142e160d P c8aa4e1129d146d4a47f70f248771868 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e6cbb5d02dcb40edb34ab5d41193496c (127.12.138.194:33581), c0f254aed5f64d8b8f923c83b2b067b7 (127.12.138.195:42421)
I20260811 02:51:58.575589 15523 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:51:58.571985 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.138.193:32817 (local address 127.12.138.195:42421)
0811 02:51:58.572106 (+ 121us) server_negotiation.cc:206] Beginning negotiation
0811 02:51:58.572111 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 02:51:58.572348 (+ 237us) server_negotiation.cc:407] Connection header received
0811 02:51:58.572447 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:51:58.572449 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:51:58.572517 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:51:58.572593 (+ 76us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 02:51:58.572900 (+ 307us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:51:58.573484 (+ 584us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:51:58.574576 (+ 1092us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:51:58.574990 (+ 414us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:51:58.575118 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:51:58.575173 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0811 02:51:58.575378 (+ 205us) server_negotiation.cc:299] Negotiation successful
0811 02:51:58.575452 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260811 02:51:58.575603 15519 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:51:58.571829 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.138.193:36669 (local address 127.12.138.194:33581)
0811 02:51:58.571984 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0811 02:51:58.571989 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 02:51:58.572238 (+ 249us) server_negotiation.cc:407] Connection header received
0811 02:51:58.572362 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:51:58.572366 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:51:58.572425 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:51:58.572504 (+ 79us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 02:51:58.572823 (+ 319us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:51:58.573402 (+ 579us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:51:58.574620 (+ 1218us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:51:58.575000 (+ 380us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:51:58.575133 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:51:58.575207 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0811 02:51:58.575423 (+ 216us) server_negotiation.cc:299] Negotiation successful
0811 02:51:58.575503 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260811 02:51:58.575618 15534 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:51:58.571705 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.138.194:33581 (local address 127.12.138.193:36669)
0811 02:51:58.572110 (+ 405us) negotiation.cc:107] Waiting for socket to connect
0811 02:51:58.572125 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 02:51:58.572253 (+ 128us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:51:58.572526 (+ 273us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:51:58.572529 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:51:58.572549 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 02:51:58.572697 (+ 148us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:51:58.572704 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:51:58.573512 (+ 808us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:51:58.573518 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:51:58.574463 (+ 945us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:51:58.574473 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:51:58.575145 (+ 672us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:51:58.575149 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:51:58.575266 (+ 117us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:51:58.575289 (+ 23us) client_negotiation.cc:769] Sending connection context
0811 02:51:58.575386 (+ 97us) client_negotiation.cc:240] Negotiation successful
0811 02:51:58.575452 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":342,"mutex_wait_us":80,"thread_start_us":125,"threads_started":1}
I20260811 02:51:58.575625 15537 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:51:58.571792 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.138.195:42421 (local address 127.12.138.193:32817)
0811 02:51:58.572232 (+ 440us) negotiation.cc:107] Waiting for socket to connect
0811 02:51:58.572248 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 02:51:58.572362 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:51:58.572630 (+ 268us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:51:58.572634 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:51:58.572653 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 02:51:58.572747 (+ 94us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:51:58.572772 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:51:58.573596 (+ 824us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:51:58.573599 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:51:58.574455 (+ 856us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:51:58.574465 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:51:58.575114 (+ 649us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:51:58.575118 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:51:58.575213 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:51:58.575247 (+ 34us) client_negotiation.cc:769] Sending connection context
0811 02:51:58.575320 (+ 73us) client_negotiation.cc:240] Negotiation successful
0811 02:51:58.575425 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":357,"mutex_wait_us":17,"thread_start_us":131,"threads_started":1}
I20260811 02:51:58.576227 15280 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bd2af0cce5e14cafa4ef26d4142e160d" candidate_uuid: "c8aa4e1129d146d4a47f70f248771868" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e6cbb5d02dcb40edb34ab5d41193496c" is_pre_election: true
I20260811 02:51:58.576443 15438 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bd2af0cce5e14cafa4ef26d4142e160d" candidate_uuid: "c8aa4e1129d146d4a47f70f248771868" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c0f254aed5f64d8b8f923c83b2b067b7" is_pre_election: true
I20260811 02:51:58.576575 15280 raft_consensus.cc:2454] T bd2af0cce5e14cafa4ef26d4142e160d P e6cbb5d02dcb40edb34ab5d41193496c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c8aa4e1129d146d4a47f70f248771868 in term 0.
I20260811 02:51:58.576766 15438 raft_consensus.cc:2454] T bd2af0cce5e14cafa4ef26d4142e160d P c0f254aed5f64d8b8f923c83b2b067b7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c8aa4e1129d146d4a47f70f248771868 in term 0.
I20260811 02:51:58.577096 15036 leader_election.cc:302] T bd2af0cce5e14cafa4ef26d4142e160d P c8aa4e1129d146d4a47f70f248771868 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c8aa4e1129d146d4a47f70f248771868, e6cbb5d02dcb40edb34ab5d41193496c; no voters:
I20260811 02:51:58.577433 15532 raft_consensus.cc:2781] T bd2af0cce5e14cafa4ef26d4142e160d P c8aa4e1129d146d4a47f70f248771868 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 02:51:58.577592 15532 raft_consensus.cc:482] T bd2af0cce5e14cafa4ef26d4142e160d P c8aa4e1129d146d4a47f70f248771868 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 02:51:58.577672 15532 raft_consensus.cc:3037] T bd2af0cce5e14cafa4ef26d4142e160d P c8aa4e1129d146d4a47f70f248771868 [term 0 FOLLOWER]: Advancing to term 1
I20260811 02:51:58.578795 15532 raft_consensus.cc:504] T bd2af0cce5e14cafa4ef26d4142e160d P c8aa4e1129d146d4a47f70f248771868 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6cbb5d02dcb40edb34ab5d41193496c" member_type: VOTER last_known_addr { host: "127.12.138.194" port: 33581 } } peers { permanent_uuid: "c8aa4e1129d146d4a47f70f248771868" member_type: VOTER last_known_addr { host: "127.12.138.193" port: 38359 } } peers { permanent_uuid: "c0f254aed5f64d8b8f923c83b2b067b7" member_type: VOTER last_known_addr { host: "127.12.138.195" port: 42421 } }
I20260811 02:51:58.579120 15532 leader_election.cc:288] T bd2af0cce5e14cafa4ef26d4142e160d P c8aa4e1129d146d4a47f70f248771868 [CANDIDATE]: Term 1 election: Requested vote from peers e6cbb5d02dcb40edb34ab5d41193496c (127.12.138.194:33581), c0f254aed5f64d8b8f923c83b2b067b7 (127.12.138.195:42421)
I20260811 02:51:58.579519 15280 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bd2af0cce5e14cafa4ef26d4142e160d" candidate_uuid: "c8aa4e1129d146d4a47f70f248771868" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e6cbb5d02dcb40edb34ab5d41193496c"
I20260811 02:51:58.579509 15438 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bd2af0cce5e14cafa4ef26d4142e160d" candidate_uuid: "c8aa4e1129d146d4a47f70f248771868" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c0f254aed5f64d8b8f923c83b2b067b7"
I20260811 02:51:58.579638 15438 raft_consensus.cc:3037] T bd2af0cce5e14cafa4ef26d4142e160d P c0f254aed5f64d8b8f923c83b2b067b7 [term 0 FOLLOWER]: Advancing to term 1
I20260811 02:51:58.579676 15280 raft_consensus.cc:3037] T bd2af0cce5e14cafa4ef26d4142e160d P e6cbb5d02dcb40edb34ab5d41193496c [term 0 FOLLOWER]: Advancing to term 1
I20260811 02:51:58.581000 15438 raft_consensus.cc:2454] T bd2af0cce5e14cafa4ef26d4142e160d P c0f254aed5f64d8b8f923c83b2b067b7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c8aa4e1129d146d4a47f70f248771868 in term 1.
I20260811 02:51:58.581000 15280 raft_consensus.cc:2454] T bd2af0cce5e14cafa4ef26d4142e160d P e6cbb5d02dcb40edb34ab5d41193496c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c8aa4e1129d146d4a47f70f248771868 in term 1.
I20260811 02:51:58.581338 15038 leader_election.cc:302] T bd2af0cce5e14cafa4ef26d4142e160d P c8aa4e1129d146d4a47f70f248771868 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c0f254aed5f64d8b8f923c83b2b067b7, c8aa4e1129d146d4a47f70f248771868; no voters:
I20260811 02:51:58.581703 15532 raft_consensus.cc:2781] T bd2af0cce5e14cafa4ef26d4142e160d P c8aa4e1129d146d4a47f70f248771868 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 02:51:58.581969 15532 raft_consensus.cc:686] T bd2af0cce5e14cafa4ef26d4142e160d P c8aa4e1129d146d4a47f70f248771868 [term 1 LEADER]: Becoming Leader. State: Replica: c8aa4e1129d146d4a47f70f248771868, State: Running, Role: LEADER
I20260811 02:51:58.582316 15532 consensus_queue.cc:237] T bd2af0cce5e14cafa4ef26d4142e160d P c8aa4e1129d146d4a47f70f248771868 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6cbb5d02dcb40edb34ab5d41193496c" member_type: VOTER last_known_addr { host: "127.12.138.194" port: 33581 } } peers { permanent_uuid: "c8aa4e1129d146d4a47f70f248771868" member_type: VOTER last_known_addr { host: "127.12.138.193" port: 38359 } } peers { permanent_uuid: "c0f254aed5f64d8b8f923c83b2b067b7" member_type: VOTER last_known_addr { host: "127.12.138.195" port: 42421 } }
I20260811 02:51:58.586004 14928 catalog_manager.cc:5495] T bd2af0cce5e14cafa4ef26d4142e160d P c8aa4e1129d146d4a47f70f248771868 reported cstate change: term changed from 0 to 1, leader changed from <none> to c8aa4e1129d146d4a47f70f248771868 (127.12.138.193). New cstate: current_term: 1 leader_uuid: "c8aa4e1129d146d4a47f70f248771868" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6cbb5d02dcb40edb34ab5d41193496c" member_type: VOTER last_known_addr { host: "127.12.138.194" port: 33581 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c8aa4e1129d146d4a47f70f248771868" member_type: VOTER last_known_addr { host: "127.12.138.193" port: 38359 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c0f254aed5f64d8b8f923c83b2b067b7" member_type: VOTER last_known_addr { host: "127.12.138.195" port: 42421 } health_report { overall_health: UNKNOWN } } }
W20260811 02:51:58.599606 15170 tablet.cc:2369] T bd2af0cce5e14cafa4ef26d4142e160d P c8aa4e1129d146d4a47f70f248771868: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 02:51:58.615909 15500 tablet.cc:2369] T bd2af0cce5e14cafa4ef26d4142e160d P c0f254aed5f64d8b8f923c83b2b067b7: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 02:51:58.631497 15333 tablet.cc:2369] T bd2af0cce5e14cafa4ef26d4142e160d P e6cbb5d02dcb40edb34ab5d41193496c: Can't schedule compaction. Clean time has not been advanced past its initial value.
Aug 11 02:51:58 dist-test-slave-9gcw krb5kdc[14879](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416717, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.138.254@KRBTEST.COM for kudu/127.12.138.254@KRBTEST.COM
I20260811 02:51:58.644164 15545 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:51:58.635157 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.138.254:41515 (local address 127.0.0.1:50226)
0811 02:51:58.635680 (+ 523us) negotiation.cc:107] Waiting for socket to connect
0811 02:51:58.635695 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 02:51:58.635811 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:51:58.636057 (+ 246us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:51:58.636059 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:51:58.636459 (+ 400us) client_negotiation.cc:189] Negotiated authn=SASL
0811 02:51:58.636559 (+ 100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:51:58.636565 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:51:58.637418 (+ 853us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:51:58.637421 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:51:58.638073 (+ 652us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:51:58.638084 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:51:58.638539 (+ 455us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:51:58.638542 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:51:58.638614 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:51:58.639326 (+ 712us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 02:51:58.639338 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0811 02:51:58.640854 (+ 1516us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 02:51:58.642459 (+ 1605us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:51:58.642469 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:51:58.642474 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:51:58.642726 (+ 252us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:51:58.643012 (+ 286us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:51:58.643015 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:51:58.643017 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:51:58.643118 (+ 101us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:51:58.643521 (+ 403us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 02:51:58.643527 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 02:51:58.643729 (+ 202us) client_negotiation.cc:769] Sending connection context
0811 02:51:58.643851 (+ 122us) client_negotiation.cc:240] Negotiation successful
0811 02:51:58.644003 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":253,"thread_start_us":131,"threads_started":1}
I20260811 02:51:58.644203 14989 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:51:58.635231 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50226 (local address 127.12.138.254:41515)
0811 02:51:58.635394 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0811 02:51:58.635399 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 02:51:58.635758 (+ 359us) server_negotiation.cc:407] Connection header received
0811 02:51:58.635897 (+ 139us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:51:58.635899 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:51:58.635942 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:51:58.636014 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:51:58.636696 (+ 682us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:51:58.637297 (+ 601us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:51:58.638195 (+ 898us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:51:58.638415 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:51:58.638493 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:51:58.641000 (+ 2507us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:51:58.641003 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:51:58.641005 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:51:58.641027 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:51:58.642357 (+ 1330us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:51:58.642814 (+ 457us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:51:58.642817 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:51:58.642818 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:51:58.642893 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:51:58.643222 (+ 329us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:51:58.643224 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:51:58.643226 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:51:58.643414 (+ 188us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:51:58.643490 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0811 02:51:58.643923 (+ 433us) server_negotiation.cc:299] Negotiation successful
0811 02:51:58.644090 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":46,"server-negotiator.queue_time_us":53}
I20260811 02:51:58.648762 14928 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.12.138.254@KRBTEST.COM'} at 127.0.0.1:50226:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260811 02:51:58.649561 14928 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 02:51:58.655051 15258 tablet_service.cc:1467] Processing CreateTablet for tablet 2cf637185b384690bab71fcc2c875f53 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=7f1f1019bd4a4a4b9594d7ed63df4882]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260811 02:51:58.655035 15091 tablet_service.cc:1467] Processing CreateTablet for tablet 2cf637185b384690bab71fcc2c875f53 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=7f1f1019bd4a4a4b9594d7ed63df4882]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260811 02:51:58.655257 15421 tablet_service.cc:1467] Processing CreateTablet for tablet 2cf637185b384690bab71fcc2c875f53 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=7f1f1019bd4a4a4b9594d7ed63df4882]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260811 02:51:58.655890 15258 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2cf637185b384690bab71fcc2c875f53. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 02:51:58.655936 15421 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2cf637185b384690bab71fcc2c875f53. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 02:51:58.655931 15091 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2cf637185b384690bab71fcc2c875f53. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 02:51:58.658811 15524 tablet_bootstrap.cc:492] T 2cf637185b384690bab71fcc2c875f53 P e6cbb5d02dcb40edb34ab5d41193496c: Bootstrap starting.
I20260811 02:51:58.659209 15525 tablet_bootstrap.cc:492] T 2cf637185b384690bab71fcc2c875f53 P c0f254aed5f64d8b8f923c83b2b067b7: Bootstrap starting.
I20260811 02:51:58.659317 15526 tablet_bootstrap.cc:492] T 2cf637185b384690bab71fcc2c875f53 P c8aa4e1129d146d4a47f70f248771868: Bootstrap starting.
I20260811 02:51:58.659901 15524 tablet_bootstrap.cc:654] T 2cf637185b384690bab71fcc2c875f53 P e6cbb5d02dcb40edb34ab5d41193496c: Neither blocks nor log segments found. Creating new log.
I20260811 02:51:58.660275 15525 tablet_bootstrap.cc:654] T 2cf637185b384690bab71fcc2c875f53 P c0f254aed5f64d8b8f923c83b2b067b7: Neither blocks nor log segments found. Creating new log.
I20260811 02:51:58.660838 15526 tablet_bootstrap.cc:654] T 2cf637185b384690bab71fcc2c875f53 P c8aa4e1129d146d4a47f70f248771868: Neither blocks nor log segments found. Creating new log.
I20260811 02:51:58.661403 15524 tablet_bootstrap.cc:492] T 2cf637185b384690bab71fcc2c875f53 P e6cbb5d02dcb40edb34ab5d41193496c: No bootstrap required, opened a new log
I20260811 02:51:58.661465 15524 ts_tablet_manager.cc:1397] T 2cf637185b384690bab71fcc2c875f53 P e6cbb5d02dcb40edb34ab5d41193496c: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.001s
I20260811 02:51:58.661715 15525 tablet_bootstrap.cc:492] T 2cf637185b384690bab71fcc2c875f53 P c0f254aed5f64d8b8f923c83b2b067b7: No bootstrap required, opened a new log
I20260811 02:51:58.661816 15525 ts_tablet_manager.cc:1397] T 2cf637185b384690bab71fcc2c875f53 P c0f254aed5f64d8b8f923c83b2b067b7: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.001s
I20260811 02:51:58.662006 15524 raft_consensus.cc:348] T 2cf637185b384690bab71fcc2c875f53 P e6cbb5d02dcb40edb34ab5d41193496c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8aa4e1129d146d4a47f70f248771868" member_type: VOTER last_known_addr { host: "127.12.138.193" port: 38359 } } peers { permanent_uuid: "e6cbb5d02dcb40edb34ab5d41193496c" member_type: VOTER last_known_addr { host: "127.12.138.194" port: 33581 } } peers { permanent_uuid: "c0f254aed5f64d8b8f923c83b2b067b7" member_type: VOTER last_known_addr { host: "127.12.138.195" port: 42421 } }
I20260811 02:51:58.662137 15524 raft_consensus.cc:374] T 2cf637185b384690bab71fcc2c875f53 P e6cbb5d02dcb40edb34ab5d41193496c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 02:51:58.662179 15524 raft_consensus.cc:729] T 2cf637185b384690bab71fcc2c875f53 P e6cbb5d02dcb40edb34ab5d41193496c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e6cbb5d02dcb40edb34ab5d41193496c, State: Initialized, Role: FOLLOWER
I20260811 02:51:58.662333 15524 consensus_queue.cc:260] T 2cf637185b384690bab71fcc2c875f53 P e6cbb5d02dcb40edb34ab5d41193496c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8aa4e1129d146d4a47f70f248771868" member_type: VOTER last_known_addr { host: "127.12.138.193" port: 38359 } } peers { permanent_uuid: "e6cbb5d02dcb40edb34ab5d41193496c" member_type: VOTER last_known_addr { host: "127.12.138.194" port: 33581 } } peers { permanent_uuid: "c0f254aed5f64d8b8f923c83b2b067b7" member_type: VOTER last_known_addr { host: "127.12.138.195" port: 42421 } }
I20260811 02:51:58.662390 15526 tablet_bootstrap.cc:492] T 2cf637185b384690bab71fcc2c875f53 P c8aa4e1129d146d4a47f70f248771868: No bootstrap required, opened a new log
I20260811 02:51:58.662340 15525 raft_consensus.cc:348] T 2cf637185b384690bab71fcc2c875f53 P c0f254aed5f64d8b8f923c83b2b067b7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8aa4e1129d146d4a47f70f248771868" member_type: VOTER last_known_addr { host: "127.12.138.193" port: 38359 } } peers { permanent_uuid: "e6cbb5d02dcb40edb34ab5d41193496c" member_type: VOTER last_known_addr { host: "127.12.138.194" port: 33581 } } peers { permanent_uuid: "c0f254aed5f64d8b8f923c83b2b067b7" member_type: VOTER last_known_addr { host: "127.12.138.195" port: 42421 } }
I20260811 02:51:58.662460 15525 raft_consensus.cc:374] T 2cf637185b384690bab71fcc2c875f53 P c0f254aed5f64d8b8f923c83b2b067b7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 02:51:58.662483 15526 ts_tablet_manager.cc:1397] T 2cf637185b384690bab71fcc2c875f53 P c8aa4e1129d146d4a47f70f248771868: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.001s
I20260811 02:51:58.662514 15525 raft_consensus.cc:729] T 2cf637185b384690bab71fcc2c875f53 P c0f254aed5f64d8b8f923c83b2b067b7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c0f254aed5f64d8b8f923c83b2b067b7, State: Initialized, Role: FOLLOWER
I20260811 02:51:58.662637 15531 tablet_replica.cc:439] T 2cf637185b384690bab71fcc2c875f53 P e6cbb5d02dcb40edb34ab5d41193496c: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8aa4e1129d146d4a47f70f248771868" member_type: VOTER last_known_addr { host: "127.12.138.193" port: 38359 } } peers { permanent_uuid: "e6cbb5d02dcb40edb34ab5d41193496c" member_type: VOTER last_known_addr { host: "127.12.138.194" port: 33581 } } peers { permanent_uuid: "c0f254aed5f64d8b8f923c83b2b067b7" member_type: VOTER last_known_addr { host: "127.12.138.195" port: 42421 } } }
I20260811 02:51:58.662701 15525 consensus_queue.cc:260] T 2cf637185b384690bab71fcc2c875f53 P c0f254aed5f64d8b8f923c83b2b067b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8aa4e1129d146d4a47f70f248771868" member_type: VOTER last_known_addr { host: "127.12.138.193" port: 38359 } } peers { permanent_uuid: "e6cbb5d02dcb40edb34ab5d41193496c" member_type: VOTER last_known_addr { host: "127.12.138.194" port: 33581 } } peers { permanent_uuid: "c0f254aed5f64d8b8f923c83b2b067b7" member_type: VOTER last_known_addr { host: "127.12.138.195" port: 42421 } }
I20260811 02:51:58.662798 15531 tablet_replica.cc:442] T 2cf637185b384690bab71fcc2c875f53 P e6cbb5d02dcb40edb34ab5d41193496c: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 02:51:58.662967 15526 raft_consensus.cc:348] T 2cf637185b384690bab71fcc2c875f53 P c8aa4e1129d146d4a47f70f248771868 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8aa4e1129d146d4a47f70f248771868" member_type: VOTER last_known_addr { host: "127.12.138.193" port: 38359 } } peers { permanent_uuid: "e6cbb5d02dcb40edb34ab5d41193496c" member_type: VOTER last_known_addr { host: "127.12.138.194" port: 33581 } } peers { permanent_uuid: "c0f254aed5f64d8b8f923c83b2b067b7" member_type: VOTER last_known_addr { host: "127.12.138.195" port: 42421 } }
I20260811 02:51:58.663093 15526 raft_consensus.cc:374] T 2cf637185b384690bab71fcc2c875f53 P c8aa4e1129d146d4a47f70f248771868 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 02:51:58.663149 15526 raft_consensus.cc:729] T 2cf637185b384690bab71fcc2c875f53 P c8aa4e1129d146d4a47f70f248771868 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c8aa4e1129d146d4a47f70f248771868, State: Initialized, Role: FOLLOWER
I20260811 02:51:58.663015 15525 ts_tablet_manager.cc:1428] T 2cf637185b384690bab71fcc2c875f53 P c0f254aed5f64d8b8f923c83b2b067b7: Time spent starting tablet: real 0.001s user 0.001s sys 0.001s
I20260811 02:51:58.663060 15533 tablet_replica.cc:439] T 2cf637185b384690bab71fcc2c875f53 P c0f254aed5f64d8b8f923c83b2b067b7: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8aa4e1129d146d4a47f70f248771868" member_type: VOTER last_known_addr { host: "127.12.138.193" port: 38359 } } peers { permanent_uuid: "e6cbb5d02dcb40edb34ab5d41193496c" member_type: VOTER last_known_addr { host: "127.12.138.194" port: 33581 } } peers { permanent_uuid: "c0f254aed5f64d8b8f923c83b2b067b7" member_type: VOTER last_known_addr { host: "127.12.138.195" port: 42421 } } }
I20260811 02:51:58.663357 15533 tablet_replica.cc:442] T 2cf637185b384690bab71fcc2c875f53 P c0f254aed5f64d8b8f923c83b2b067b7: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 02:51:58.663601 15524 ts_tablet_manager.cc:1428] T 2cf637185b384690bab71fcc2c875f53 P e6cbb5d02dcb40edb34ab5d41193496c: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20260811 02:51:58.663782 15526 consensus_queue.cc:260] T 2cf637185b384690bab71fcc2c875f53 P c8aa4e1129d146d4a47f70f248771868 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8aa4e1129d146d4a47f70f248771868" member_type: VOTER last_known_addr { host: "127.12.138.193" port: 38359 } } peers { permanent_uuid: "e6cbb5d02dcb40edb34ab5d41193496c" member_type: VOTER last_known_addr { host: "127.12.138.194" port: 33581 } } peers { permanent_uuid: "c0f254aed5f64d8b8f923c83b2b067b7" member_type: VOTER last_known_addr { host: "127.12.138.195" port: 42421 } }
I20260811 02:51:58.664126 15532 tablet_replica.cc:439] T 2cf637185b384690bab71fcc2c875f53 P c8aa4e1129d146d4a47f70f248771868: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8aa4e1129d146d4a47f70f248771868" member_type: VOTER last_known_addr { host: "127.12.138.193" port: 38359 } } peers { permanent_uuid: "e6cbb5d02dcb40edb34ab5d41193496c" member_type: VOTER last_known_addr { host: "127.12.138.194" port: 33581 } } peers { permanent_uuid: "c0f254aed5f64d8b8f923c83b2b067b7" member_type: VOTER last_known_addr { host: "127.12.138.195" port: 42421 } } }
I20260811 02:51:58.664269 15532 tablet_replica.cc:442] T 2cf637185b384690bab71fcc2c875f53 P c8aa4e1129d146d4a47f70f248771868: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 02:51:58.665087 15526 ts_tablet_manager.cc:1428] T 2cf637185b384690bab71fcc2c875f53 P c8aa4e1129d146d4a47f70f248771868: Time spent starting tablet: real 0.003s user 0.001s sys 0.001s
I20260811 02:51:58.715734 15532 raft_consensus.cc:482] T 2cf637185b384690bab71fcc2c875f53 P c8aa4e1129d146d4a47f70f248771868 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 02:51:58.715880 15532 raft_consensus.cc:504] T 2cf637185b384690bab71fcc2c875f53 P c8aa4e1129d146d4a47f70f248771868 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8aa4e1129d146d4a47f70f248771868" member_type: VOTER last_known_addr { host: "127.12.138.193" port: 38359 } } peers { permanent_uuid: "e6cbb5d02dcb40edb34ab5d41193496c" member_type: VOTER last_known_addr { host: "127.12.138.194" port: 33581 } } peers { permanent_uuid: "c0f254aed5f64d8b8f923c83b2b067b7" member_type: VOTER last_known_addr { host: "127.12.138.195" port: 42421 } }
I20260811 02:51:58.716365 15532 leader_election.cc:288] T 2cf637185b384690bab71fcc2c875f53 P c8aa4e1129d146d4a47f70f248771868 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e6cbb5d02dcb40edb34ab5d41193496c (127.12.138.194:33581), c0f254aed5f64d8b8f923c83b2b067b7 (127.12.138.195:42421)
I20260811 02:51:58.716852 15280 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2cf637185b384690bab71fcc2c875f53" candidate_uuid: "c8aa4e1129d146d4a47f70f248771868" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e6cbb5d02dcb40edb34ab5d41193496c" is_pre_election: true
I20260811 02:51:58.716895 15438 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2cf637185b384690bab71fcc2c875f53" candidate_uuid: "c8aa4e1129d146d4a47f70f248771868" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c0f254aed5f64d8b8f923c83b2b067b7" is_pre_election: true
I20260811 02:51:58.716990 15280 raft_consensus.cc:2454] T 2cf637185b384690bab71fcc2c875f53 P e6cbb5d02dcb40edb34ab5d41193496c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c8aa4e1129d146d4a47f70f248771868 in term 0.
I20260811 02:51:58.717110 15438 raft_consensus.cc:2454] T 2cf637185b384690bab71fcc2c875f53 P c0f254aed5f64d8b8f923c83b2b067b7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c8aa4e1129d146d4a47f70f248771868 in term 0.
I20260811 02:51:58.717363 15036 leader_election.cc:302] T 2cf637185b384690bab71fcc2c875f53 P c8aa4e1129d146d4a47f70f248771868 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c8aa4e1129d146d4a47f70f248771868, e6cbb5d02dcb40edb34ab5d41193496c; no voters:
I20260811 02:51:58.717585 15532 raft_consensus.cc:2781] T 2cf637185b384690bab71fcc2c875f53 P c8aa4e1129d146d4a47f70f248771868 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 02:51:58.717654 15532 raft_consensus.cc:482] T 2cf637185b384690bab71fcc2c875f53 P c8aa4e1129d146d4a47f70f248771868 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 02:51:58.717697 15532 raft_consensus.cc:3037] T 2cf637185b384690bab71fcc2c875f53 P c8aa4e1129d146d4a47f70f248771868 [term 0 FOLLOWER]: Advancing to term 1
I20260811 02:51:58.718791 15532 raft_consensus.cc:504] T 2cf637185b384690bab71fcc2c875f53 P c8aa4e1129d146d4a47f70f248771868 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8aa4e1129d146d4a47f70f248771868" member_type: VOTER last_known_addr { host: "127.12.138.193" port: 38359 } } peers { permanent_uuid: "e6cbb5d02dcb40edb34ab5d41193496c" member_type: VOTER last_known_addr { host: "127.12.138.194" port: 33581 } } peers { permanent_uuid: "c0f254aed5f64d8b8f923c83b2b067b7" member_type: VOTER last_known_addr { host: "127.12.138.195" port: 42421 } }
I20260811 02:51:58.719121 15532 leader_election.cc:288] T 2cf637185b384690bab71fcc2c875f53 P c8aa4e1129d146d4a47f70f248771868 [CANDIDATE]: Term 1 election: Requested vote from peers e6cbb5d02dcb40edb34ab5d41193496c (127.12.138.194:33581), c0f254aed5f64d8b8f923c83b2b067b7 (127.12.138.195:42421)
I20260811 02:51:58.719478 15280 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2cf637185b384690bab71fcc2c875f53" candidate_uuid: "c8aa4e1129d146d4a47f70f248771868" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e6cbb5d02dcb40edb34ab5d41193496c"
I20260811 02:51:58.719544 15438 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2cf637185b384690bab71fcc2c875f53" candidate_uuid: "c8aa4e1129d146d4a47f70f248771868" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c0f254aed5f64d8b8f923c83b2b067b7"
I20260811 02:51:58.719624 15280 raft_consensus.cc:3037] T 2cf637185b384690bab71fcc2c875f53 P e6cbb5d02dcb40edb34ab5d41193496c [term 0 FOLLOWER]: Advancing to term 1
I20260811 02:51:58.719691 15438 raft_consensus.cc:3037] T 2cf637185b384690bab71fcc2c875f53 P c0f254aed5f64d8b8f923c83b2b067b7 [term 0 FOLLOWER]: Advancing to term 1
I20260811 02:51:58.720646 15280 raft_consensus.cc:2454] T 2cf637185b384690bab71fcc2c875f53 P e6cbb5d02dcb40edb34ab5d41193496c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c8aa4e1129d146d4a47f70f248771868 in term 1.
I20260811 02:51:58.720718 15438 raft_consensus.cc:2454] T 2cf637185b384690bab71fcc2c875f53 P c0f254aed5f64d8b8f923c83b2b067b7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c8aa4e1129d146d4a47f70f248771868 in term 1.
I20260811 02:51:58.720965 15036 leader_election.cc:302] T 2cf637185b384690bab71fcc2c875f53 P c8aa4e1129d146d4a47f70f248771868 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c8aa4e1129d146d4a47f70f248771868, e6cbb5d02dcb40edb34ab5d41193496c; no voters:
I20260811 02:51:58.721179 15532 raft_consensus.cc:2781] T 2cf637185b384690bab71fcc2c875f53 P c8aa4e1129d146d4a47f70f248771868 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 02:51:58.721271 15532 raft_consensus.cc:686] T 2cf637185b384690bab71fcc2c875f53 P c8aa4e1129d146d4a47f70f248771868 [term 1 LEADER]: Becoming Leader. State: Replica: c8aa4e1129d146d4a47f70f248771868, State: Running, Role: LEADER
I20260811 02:51:58.721405 15532 consensus_queue.cc:237] T 2cf637185b384690bab71fcc2c875f53 P c8aa4e1129d146d4a47f70f248771868 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8aa4e1129d146d4a47f70f248771868" member_type: VOTER last_known_addr { host: "127.12.138.193" port: 38359 } } peers { permanent_uuid: "e6cbb5d02dcb40edb34ab5d41193496c" member_type: VOTER last_known_addr { host: "127.12.138.194" port: 33581 } } peers { permanent_uuid: "c0f254aed5f64d8b8f923c83b2b067b7" member_type: VOTER last_known_addr { host: "127.12.138.195" port: 42421 } }
I20260811 02:51:58.722199 15538 tablet_replica.cc:439] T 2cf637185b384690bab71fcc2c875f53 P c8aa4e1129d146d4a47f70f248771868: TxnStatusTablet state changed. Reason: New leader c8aa4e1129d146d4a47f70f248771868. Latest consensus state: current_term: 1 leader_uuid: "c8aa4e1129d146d4a47f70f248771868" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8aa4e1129d146d4a47f70f248771868" member_type: VOTER last_known_addr { host: "127.12.138.193" port: 38359 } } peers { permanent_uuid: "e6cbb5d02dcb40edb34ab5d41193496c" member_type: VOTER last_known_addr { host: "127.12.138.194" port: 33581 } } peers { permanent_uuid: "c0f254aed5f64d8b8f923c83b2b067b7" member_type: VOTER last_known_addr { host: "127.12.138.195" port: 42421 } } }
I20260811 02:51:58.722309 15538 tablet_replica.cc:442] T 2cf637185b384690bab71fcc2c875f53 P c8aa4e1129d146d4a47f70f248771868: This TxnStatusTablet replica's current role is: LEADER
I20260811 02:51:58.722625 15551 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260811 02:51:58.722908 15551 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260811 02:51:58.723400 14928 catalog_manager.cc:5495] T 2cf637185b384690bab71fcc2c875f53 P c8aa4e1129d146d4a47f70f248771868 reported cstate change: term changed from 0 to 1, leader changed from <none> to c8aa4e1129d146d4a47f70f248771868 (127.12.138.193). New cstate: current_term: 1 leader_uuid: "c8aa4e1129d146d4a47f70f248771868" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8aa4e1129d146d4a47f70f248771868" member_type: VOTER last_known_addr { host: "127.12.138.193" port: 38359 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e6cbb5d02dcb40edb34ab5d41193496c" member_type: VOTER last_known_addr { host: "127.12.138.194" port: 33581 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c0f254aed5f64d8b8f923c83b2b067b7" member_type: VOTER last_known_addr { host: "127.12.138.195" port: 42421 } health_report { overall_health: UNKNOWN } } }
I20260811 02:51:58.746167 15545 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:51:58.741731 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.138.193:38359 (local address 127.0.0.1:37054)
0811 02:51:58.741904 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0811 02:51:58.741922 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 02:51:58.742029 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:51:58.742345 (+ 316us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:51:58.742348 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:51:58.742354 (+ 6us) client_negotiation.cc:189] Negotiated authn=TOKEN
0811 02:51:58.742451 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:51:58.742456 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:51:58.743521 (+ 1065us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:51:58.743525 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:51:58.744251 (+ 726us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:51:58.744263 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:51:58.744911 (+ 648us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:51:58.744914 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:51:58.744993 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:51:58.745045 (+ 52us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0811 02:51:58.745874 (+ 829us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0811 02:51:58.745897 (+ 23us) client_negotiation.cc:769] Sending connection context
0811 02:51:58.745965 (+ 68us) client_negotiation.cc:240] Negotiation successful
0811 02:51:58.746045 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":53}
I20260811 02:51:58.746212 15522 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:51:58.741893 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37054 (local address 127.12.138.193:38359)
0811 02:51:58.742042 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0811 02:51:58.742047 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 02:51:58.742064 (+ 17us) server_negotiation.cc:407] Connection header received
0811 02:51:58.742139 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:51:58.742142 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:51:58.742205 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:51:58.742311 (+ 106us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 02:51:58.742574 (+ 263us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:51:58.743384 (+ 810us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:51:58.744390 (+ 1006us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:51:58.744743 (+ 353us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:51:58.744852 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:51:58.745216 (+ 364us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 02:51:58.745754 (+ 538us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 02:51:58.745876 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0811 02:51:58.746011 (+ 135us) server_negotiation.cc:299] Negotiation successful
0811 02:51:58.746074 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260811 02:51:58.751698 15280 raft_consensus.cc:1261] T 2cf637185b384690bab71fcc2c875f53 P e6cbb5d02dcb40edb34ab5d41193496c [term 1 FOLLOWER]: Refusing update from remote peer c8aa4e1129d146d4a47f70f248771868: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 02:51:58.751739 15438 raft_consensus.cc:1261] T 2cf637185b384690bab71fcc2c875f53 P c0f254aed5f64d8b8f923c83b2b067b7 [term 1 FOLLOWER]: Refusing update from remote peer c8aa4e1129d146d4a47f70f248771868: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 02:51:58.752485 15538 consensus_queue.cc:1035] T 2cf637185b384690bab71fcc2c875f53 P c8aa4e1129d146d4a47f70f248771868 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e6cbb5d02dcb40edb34ab5d41193496c" member_type: VOTER last_known_addr { host: "127.12.138.194" port: 33581 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 02:51:58.752791 15532 consensus_queue.cc:1035] T 2cf637185b384690bab71fcc2c875f53 P c8aa4e1129d146d4a47f70f248771868 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c0f254aed5f64d8b8f923c83b2b067b7" member_type: VOTER last_known_addr { host: "127.12.138.195" port: 42421 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 02:51:58.756834 15533 tablet_replica.cc:439] T 2cf637185b384690bab71fcc2c875f53 P c0f254aed5f64d8b8f923c83b2b067b7: TxnStatusTablet state changed. Reason: New leader c8aa4e1129d146d4a47f70f248771868. Latest consensus state: current_term: 1 leader_uuid: "c8aa4e1129d146d4a47f70f248771868" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8aa4e1129d146d4a47f70f248771868" member_type: VOTER last_known_addr { host: "127.12.138.193" port: 38359 } } peers { permanent_uuid: "e6cbb5d02dcb40edb34ab5d41193496c" member_type: VOTER last_known_addr { host: "127.12.138.194" port: 33581 } } peers { permanent_uuid: "c0f254aed5f64d8b8f923c83b2b067b7" member_type: VOTER last_known_addr { host: "127.12.138.195" port: 42421 } } }
I20260811 02:51:58.756834 15531 tablet_replica.cc:439] T 2cf637185b384690bab71fcc2c875f53 P e6cbb5d02dcb40edb34ab5d41193496c: TxnStatusTablet state changed. Reason: New leader c8aa4e1129d146d4a47f70f248771868. Latest consensus state: current_term: 1 leader_uuid: "c8aa4e1129d146d4a47f70f248771868" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8aa4e1129d146d4a47f70f248771868" member_type: VOTER last_known_addr { host: "127.12.138.193" port: 38359 } } peers { permanent_uuid: "e6cbb5d02dcb40edb34ab5d41193496c" member_type: VOTER last_known_addr { host: "127.12.138.194" port: 33581 } } peers { permanent_uuid: "c0f254aed5f64d8b8f923c83b2b067b7" member_type: VOTER last_known_addr { host: "127.12.138.195" port: 42421 } } }
I20260811 02:51:58.756953 15531 tablet_replica.cc:442] T 2cf637185b384690bab71fcc2c875f53 P e6cbb5d02dcb40edb34ab5d41193496c: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 02:51:58.756953 15533 tablet_replica.cc:442] T 2cf637185b384690bab71fcc2c875f53 P c0f254aed5f64d8b8f923c83b2b067b7: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 02:51:58.760152 15532 tablet_replica.cc:439] T 2cf637185b384690bab71fcc2c875f53 P c8aa4e1129d146d4a47f70f248771868: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "c8aa4e1129d146d4a47f70f248771868" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8aa4e1129d146d4a47f70f248771868" member_type: VOTER last_known_addr { host: "127.12.138.193" port: 38359 } } peers { permanent_uuid: "e6cbb5d02dcb40edb34ab5d41193496c" member_type: VOTER last_known_addr { host: "127.12.138.194" port: 33581 } } peers { permanent_uuid: "c0f254aed5f64d8b8f923c83b2b067b7" member_type: VOTER last_known_addr { host: "127.12.138.195" port: 42421 } } }
I20260811 02:51:58.760272 15532 tablet_replica.cc:442] T 2cf637185b384690bab71fcc2c875f53 P c8aa4e1129d146d4a47f70f248771868: This TxnStatusTablet replica's current role is: LEADER
I20260811 02:51:58.761682 15538 tablet_replica.cc:439] T 2cf637185b384690bab71fcc2c875f53 P c8aa4e1129d146d4a47f70f248771868: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "c8aa4e1129d146d4a47f70f248771868" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8aa4e1129d146d4a47f70f248771868" member_type: VOTER last_known_addr { host: "127.12.138.193" port: 38359 } } peers { permanent_uuid: "e6cbb5d02dcb40edb34ab5d41193496c" member_type: VOTER last_known_addr { host: "127.12.138.194" port: 33581 } } peers { permanent_uuid: "c0f254aed5f64d8b8f923c83b2b067b7" member_type: VOTER last_known_addr { host: "127.12.138.195" port: 42421 } } }
I20260811 02:51:58.761786 15538 tablet_replica.cc:442] T 2cf637185b384690bab71fcc2c875f53 P c8aa4e1129d146d4a47f70f248771868: This TxnStatusTablet replica's current role is: LEADER
I20260811 02:51:58.762516 15531 tablet_replica.cc:439] T 2cf637185b384690bab71fcc2c875f53 P e6cbb5d02dcb40edb34ab5d41193496c: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "c8aa4e1129d146d4a47f70f248771868" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8aa4e1129d146d4a47f70f248771868" member_type: VOTER last_known_addr { host: "127.12.138.193" port: 38359 } } peers { permanent_uuid: "e6cbb5d02dcb40edb34ab5d41193496c" member_type: VOTER last_known_addr { host: "127.12.138.194" port: 33581 } } peers { permanent_uuid: "c0f254aed5f64d8b8f923c83b2b067b7" member_type: VOTER last_known_addr { host: "127.12.138.195" port: 42421 } } }
I20260811 02:51:58.762627 15531 tablet_replica.cc:442] T 2cf637185b384690bab71fcc2c875f53 P e6cbb5d02dcb40edb34ab5d41193496c: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 02:51:58.762693 15555 mvcc.cc:205] Tried to move back new op lower bound from 7317162879999000576 to 7317162879884455936. Current Snapshot: MvccSnapshot[applied={T|T < 7317162879999000576}]
I20260811 02:51:58.762861 15557 mvcc.cc:205] Tried to move back new op lower bound from 7317162879999000576 to 7317162879884455936. Current Snapshot: MvccSnapshot[applied={T|T < 7317162879999000576}]
I20260811 02:51:58.762986 15533 tablet_replica.cc:439] T 2cf637185b384690bab71fcc2c875f53 P c0f254aed5f64d8b8f923c83b2b067b7: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "c8aa4e1129d146d4a47f70f248771868" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8aa4e1129d146d4a47f70f248771868" member_type: VOTER last_known_addr { host: "127.12.138.193" port: 38359 } } peers { permanent_uuid: "e6cbb5d02dcb40edb34ab5d41193496c" member_type: VOTER last_known_addr { host: "127.12.138.194" port: 33581 } } peers { permanent_uuid: "c0f254aed5f64d8b8f923c83b2b067b7" member_type: VOTER last_known_addr { host: "127.12.138.195" port: 42421 } } }
I20260811 02:51:58.763079 15533 tablet_replica.cc:442] T 2cf637185b384690bab71fcc2c875f53 P c0f254aed5f64d8b8f923c83b2b067b7: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 02:51:58.772102 15522 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:51:58.768255 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37060 (local address 127.12.138.193:38359)
0811 02:51:58.768497 (+ 242us) server_negotiation.cc:206] Beginning negotiation
0811 02:51:58.768502 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 02:51:58.768541 (+ 39us) server_negotiation.cc:407] Connection header received
0811 02:51:58.768687 (+ 146us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:51:58.768690 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:51:58.768740 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:51:58.768846 (+ 106us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 02:51:58.769271 (+ 425us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:51:58.770054 (+ 783us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:51:58.770960 (+ 906us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:51:58.771296 (+ 336us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:51:58.771410 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:51:58.771633 (+ 223us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 02:51:58.771714 (+ 81us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 02:51:58.771785 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0811 02:51:58.771910 (+ 125us) server_negotiation.cc:299] Negotiation successful
0811 02:51:58.771964 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":158}
I20260811 02:51:58.783761 15522 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:51:58.780337 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.138.193:36433 (local address 127.12.138.193:38359)
0811 02:51:58.780457 (+ 120us) server_negotiation.cc:206] Beginning negotiation
0811 02:51:58.780462 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 02:51:58.780685 (+ 223us) server_negotiation.cc:407] Connection header received
0811 02:51:58.780850 (+ 165us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:51:58.780853 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:51:58.780897 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:51:58.780977 (+ 80us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 02:51:58.781249 (+ 272us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:51:58.781743 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:51:58.782601 (+ 858us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:51:58.782835 (+ 234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:51:58.782972 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:51:58.783292 (+ 320us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 02:51:58.783364 (+ 72us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 02:51:58.783447 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0811 02:51:58.783534 (+ 87us) server_negotiation.cc:299] Negotiation successful
0811 02:51:58.783584 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260811 02:51:58.783775 15566 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:51:58.780264 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.138.193:38359 (local address 127.12.138.193:36433)
0811 02:51:58.780557 (+ 293us) negotiation.cc:107] Waiting for socket to connect
0811 02:51:58.780569 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 02:51:58.780741 (+ 172us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:51:58.780983 (+ 242us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:51:58.780986 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:51:58.780992 (+ 6us) client_negotiation.cc:189] Negotiated authn=TOKEN
0811 02:51:58.781107 (+ 115us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:51:58.781123 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:51:58.781853 (+ 730us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:51:58.781859 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:51:58.782481 (+ 622us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:51:58.782488 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:51:58.782982 (+ 494us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:51:58.782985 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:51:58.783086 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:51:58.783136 (+ 50us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0811 02:51:58.783452 (+ 316us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0811 02:51:58.783459 (+ 7us) client_negotiation.cc:769] Sending connection context
0811 02:51:58.783525 (+ 66us) client_negotiation.cc:240] Negotiation successful
0811 02:51:58.783599 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":210,"mutex_wait_us":1,"thread_start_us":128,"threads_started":1}
I20260811 02:51:58.791548 15280 raft_consensus.cc:1261] T bd2af0cce5e14cafa4ef26d4142e160d P e6cbb5d02dcb40edb34ab5d41193496c [term 1 FOLLOWER]: Refusing update from remote peer c8aa4e1129d146d4a47f70f248771868: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 02:51:58.791566 15438 raft_consensus.cc:1261] T bd2af0cce5e14cafa4ef26d4142e160d P c0f254aed5f64d8b8f923c83b2b067b7 [term 1 FOLLOWER]: Refusing update from remote peer c8aa4e1129d146d4a47f70f248771868: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 02:51:58.791989 15538 consensus_queue.cc:1035] T bd2af0cce5e14cafa4ef26d4142e160d P c8aa4e1129d146d4a47f70f248771868 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c0f254aed5f64d8b8f923c83b2b067b7" member_type: VOTER last_known_addr { host: "127.12.138.195" port: 42421 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 02:51:58.792387 15538 consensus_queue.cc:1035] T bd2af0cce5e14cafa4ef26d4142e160d P c8aa4e1129d146d4a47f70f248771868 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e6cbb5d02dcb40edb34ab5d41193496c" member_type: VOTER last_known_addr { host: "127.12.138.194" port: 33581 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 02:51:58.797267 15554 mvcc.cc:205] Tried to move back new op lower bound from 7317162880166260736 to 7317162879315726336. Current Snapshot: MvccSnapshot[applied={T|T < 7317162880166260736}]
I20260811 02:51:58.838662 14927 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50218:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 02:51:58.838893 14927 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50218:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 02:51:58.841571 14927 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 60ef762b8fc246488dfb261f0b1e8420: Sending DeleteTablet for 3 replicas of tablet bd2af0cce5e14cafa4ef26d4142e160d
I20260811 02:51:58.842324 15258 tablet_service.cc:1514] Processing DeleteTablet for tablet bd2af0cce5e14cafa4ef26d4142e160d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 02:51:58 UTC) from {username='kudu', principal='kudu/127.12.138.254@KRBTEST.COM'} at 127.0.0.1:53890
I20260811 02:51:58.842339 15090 tablet_service.cc:1514] Processing DeleteTablet for tablet bd2af0cce5e14cafa4ef26d4142e160d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 02:51:58 UTC) from {username='kudu', principal='kudu/127.12.138.254@KRBTEST.COM'} at 127.0.0.1:37042
I20260811 02:51:58.842881 15573 tablet_replica.cc:330] T bd2af0cce5e14cafa4ef26d4142e160d P e6cbb5d02dcb40edb34ab5d41193496c: stopping tablet replica
I20260811 02:51:58.842844 15421 tablet_service.cc:1514] Processing DeleteTablet for tablet bd2af0cce5e14cafa4ef26d4142e160d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 02:51:58 UTC) from {username='kudu', principal='kudu/127.12.138.254@KRBTEST.COM'} at 127.0.0.1:40906
I20260811 02:51:58.843235 15573 raft_consensus.cc:2229] T bd2af0cce5e14cafa4ef26d4142e160d P e6cbb5d02dcb40edb34ab5d41193496c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 02:51:58.843477 15573 raft_consensus.cc:2258] T bd2af0cce5e14cafa4ef26d4142e160d P e6cbb5d02dcb40edb34ab5d41193496c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 02:51:58.843597 15575 tablet_replica.cc:330] T bd2af0cce5e14cafa4ef26d4142e160d P c0f254aed5f64d8b8f923c83b2b067b7: stopping tablet replica
I20260811 02:51:58.843582 14927 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50218:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 02:51:58.843863 15575 raft_consensus.cc:2229] T bd2af0cce5e14cafa4ef26d4142e160d P c0f254aed5f64d8b8f923c83b2b067b7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 02:51:58.843871 15574 tablet_replica.cc:330] T bd2af0cce5e14cafa4ef26d4142e160d P c8aa4e1129d146d4a47f70f248771868: stopping tablet replica
I20260811 02:51:58.844039 15575 raft_consensus.cc:2258] T bd2af0cce5e14cafa4ef26d4142e160d P c0f254aed5f64d8b8f923c83b2b067b7 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260811 02:51:58.844079 14927 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 02:51:58.844159 15574 raft_consensus.cc:2229] T bd2af0cce5e14cafa4ef26d4142e160d P c8aa4e1129d146d4a47f70f248771868 [term 1 LEADER]: Raft consensus shutting down.
I20260811 02:51:58.844494 15574 raft_consensus.cc:2258] T bd2af0cce5e14cafa4ef26d4142e160d P c8aa4e1129d146d4a47f70f248771868 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 02:51:58.845652 15575 ts_tablet_manager.cc:1905] T bd2af0cce5e14cafa4ef26d4142e160d P c0f254aed5f64d8b8f923c83b2b067b7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 02:51:58.846181 15574 ts_tablet_manager.cc:1905] T bd2af0cce5e14cafa4ef26d4142e160d P c8aa4e1129d146d4a47f70f248771868: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 02:51:58.848771 15575 ts_tablet_manager.cc:1918] T bd2af0cce5e14cafa4ef26d4142e160d P c0f254aed5f64d8b8f923c83b2b067b7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 02:51:58.848866 15575 log.cc:1196] T bd2af0cce5e14cafa4ef26d4142e160d P c0f254aed5f64d8b8f923c83b2b067b7: Deleting WAL directory at /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786416707705494-12843-0/minicluster-data/ts-2/wal/wals/bd2af0cce5e14cafa4ef26d4142e160d
I20260811 02:51:58.849205 15575 ts_tablet_manager.cc:1939] T bd2af0cce5e14cafa4ef26d4142e160d P c0f254aed5f64d8b8f923c83b2b067b7: Deleting consensus metadata
I20260811 02:51:58.849634 15574 ts_tablet_manager.cc:1918] T bd2af0cce5e14cafa4ef26d4142e160d P c8aa4e1129d146d4a47f70f248771868: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 02:51:58.849735 15574 log.cc:1196] T bd2af0cce5e14cafa4ef26d4142e160d P c8aa4e1129d146d4a47f70f248771868: Deleting WAL directory at /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786416707705494-12843-0/minicluster-data/ts-0/wal/wals/bd2af0cce5e14cafa4ef26d4142e160d
I20260811 02:51:58.850101 15574 ts_tablet_manager.cc:1939] T bd2af0cce5e14cafa4ef26d4142e160d P c8aa4e1129d146d4a47f70f248771868: Deleting consensus metadata
I20260811 02:51:58.851238 14919 catalog_manager.cc:4841] TS c0f254aed5f64d8b8f923c83b2b067b7 (127.12.138.195:42421): tablet bd2af0cce5e14cafa4ef26d4142e160d (table test-table [id=a70705a1c7c04407b51153325661c6fc]) successfully deleted
I20260811 02:51:58.851516 14918 catalog_manager.cc:4841] TS c8aa4e1129d146d4a47f70f248771868 (127.12.138.193:38359): tablet bd2af0cce5e14cafa4ef26d4142e160d (table test-table [id=a70705a1c7c04407b51153325661c6fc]) successfully deleted
I20260811 02:51:58.854415 15090 tablet_service.cc:1467] Processing CreateTablet for tablet e5f89aecee624e30b430ac9329c17d4a (DEFAULT_TABLE table=test-table [id=a19b76a9f24643c787f2433fb034ddb8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 02:51:58.854462 15573 ts_tablet_manager.cc:1905] T bd2af0cce5e14cafa4ef26d4142e160d P e6cbb5d02dcb40edb34ab5d41193496c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 02:51:58.854727 15090 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e5f89aecee624e30b430ac9329c17d4a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 02:51:58.856657 15526 tablet_bootstrap.cc:492] T e5f89aecee624e30b430ac9329c17d4a P c8aa4e1129d146d4a47f70f248771868: Bootstrap starting.
I20260811 02:51:58.857708 15421 tablet_service.cc:1467] Processing CreateTablet for tablet e5f89aecee624e30b430ac9329c17d4a (DEFAULT_TABLE table=test-table [id=a19b76a9f24643c787f2433fb034ddb8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 02:51:58.857852 15573 ts_tablet_manager.cc:1918] T bd2af0cce5e14cafa4ef26d4142e160d P e6cbb5d02dcb40edb34ab5d41193496c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 02:51:58.857960 15573 log.cc:1196] T bd2af0cce5e14cafa4ef26d4142e160d P e6cbb5d02dcb40edb34ab5d41193496c: Deleting WAL directory at /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786416707705494-12843-0/minicluster-data/ts-1/wal/wals/bd2af0cce5e14cafa4ef26d4142e160d
I20260811 02:51:58.858042 15421 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e5f89aecee624e30b430ac9329c17d4a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 02:51:58.858309 15573 ts_tablet_manager.cc:1939] T bd2af0cce5e14cafa4ef26d4142e160d P e6cbb5d02dcb40edb34ab5d41193496c: Deleting consensus metadata
I20260811 02:51:58.859620 14918 catalog_manager.cc:4841] TS e6cbb5d02dcb40edb34ab5d41193496c (127.12.138.194:33581): tablet bd2af0cce5e14cafa4ef26d4142e160d (table test-table [id=a70705a1c7c04407b51153325661c6fc]) successfully deleted
I20260811 02:51:58.860345 15525 tablet_bootstrap.cc:492] T e5f89aecee624e30b430ac9329c17d4a P c0f254aed5f64d8b8f923c83b2b067b7: Bootstrap starting.
I20260811 02:51:58.860646 15258 tablet_service.cc:1467] Processing CreateTablet for tablet e5f89aecee624e30b430ac9329c17d4a (DEFAULT_TABLE table=test-table [id=a19b76a9f24643c787f2433fb034ddb8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 02:51:58.860903 15258 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e5f89aecee624e30b430ac9329c17d4a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 02:51:58.861330 15525 tablet_bootstrap.cc:654] T e5f89aecee624e30b430ac9329c17d4a P c0f254aed5f64d8b8f923c83b2b067b7: Neither blocks nor log segments found. Creating new log.
I20260811 02:51:58.862962 15524 tablet_bootstrap.cc:492] T e5f89aecee624e30b430ac9329c17d4a P e6cbb5d02dcb40edb34ab5d41193496c: Bootstrap starting.
I20260811 02:51:58.863512 15526 tablet_bootstrap.cc:654] T e5f89aecee624e30b430ac9329c17d4a P c8aa4e1129d146d4a47f70f248771868: Neither blocks nor log segments found. Creating new log.
I20260811 02:51:58.863915 15524 tablet_bootstrap.cc:654] T e5f89aecee624e30b430ac9329c17d4a P e6cbb5d02dcb40edb34ab5d41193496c: Neither blocks nor log segments found. Creating new log.
I20260811 02:51:58.865494 15526 tablet_bootstrap.cc:492] T e5f89aecee624e30b430ac9329c17d4a P c8aa4e1129d146d4a47f70f248771868: No bootstrap required, opened a new log
I20260811 02:51:58.865592 15525 tablet_bootstrap.cc:492] T e5f89aecee624e30b430ac9329c17d4a P c0f254aed5f64d8b8f923c83b2b067b7: No bootstrap required, opened a new log
I20260811 02:51:58.865585 15526 ts_tablet_manager.cc:1397] T e5f89aecee624e30b430ac9329c17d4a P c8aa4e1129d146d4a47f70f248771868: Time spent bootstrapping tablet: real 0.009s user 0.001s sys 0.001s
I20260811 02:51:58.865684 15525 ts_tablet_manager.cc:1397] T e5f89aecee624e30b430ac9329c17d4a P c0f254aed5f64d8b8f923c83b2b067b7: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20260811 02:51:58.866050 15526 raft_consensus.cc:348] T e5f89aecee624e30b430ac9329c17d4a P c8aa4e1129d146d4a47f70f248771868 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8aa4e1129d146d4a47f70f248771868" member_type: VOTER last_known_addr { host: "127.12.138.193" port: 38359 } } peers { permanent_uuid: "e6cbb5d02dcb40edb34ab5d41193496c" member_type: VOTER last_known_addr { host: "127.12.138.194" port: 33581 } } peers { permanent_uuid: "c0f254aed5f64d8b8f923c83b2b067b7" member_type: VOTER last_known_addr { host: "127.12.138.195" port: 42421 } }
I20260811 02:51:58.866098 15525 raft_consensus.cc:348] T e5f89aecee624e30b430ac9329c17d4a P c0f254aed5f64d8b8f923c83b2b067b7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8aa4e1129d146d4a47f70f248771868" member_type: VOTER last_known_addr { host: "127.12.138.193" port: 38359 } } peers { permanent_uuid: "e6cbb5d02dcb40edb34ab5d41193496c" member_type: VOTER last_known_addr { host: "127.12.138.194" port: 33581 } } peers { permanent_uuid: "c0f254aed5f64d8b8f923c83b2b067b7" member_type: VOTER last_known_addr { host: "127.12.138.195" port: 42421 } }
I20260811 02:51:58.866184 15526 raft_consensus.cc:374] T e5f89aecee624e30b430ac9329c17d4a P c8aa4e1129d146d4a47f70f248771868 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 02:51:58.866199 15525 raft_consensus.cc:374] T e5f89aecee624e30b430ac9329c17d4a P c0f254aed5f64d8b8f923c83b2b067b7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 02:51:58.866225 15526 raft_consensus.cc:729] T e5f89aecee624e30b430ac9329c17d4a P c8aa4e1129d146d4a47f70f248771868 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c8aa4e1129d146d4a47f70f248771868, State: Initialized, Role: FOLLOWER
I20260811 02:51:58.866235 15525 raft_consensus.cc:729] T e5f89aecee624e30b430ac9329c17d4a P c0f254aed5f64d8b8f923c83b2b067b7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c0f254aed5f64d8b8f923c83b2b067b7, State: Initialized, Role: FOLLOWER
I20260811 02:51:58.866355 15525 consensus_queue.cc:260] T e5f89aecee624e30b430ac9329c17d4a P c0f254aed5f64d8b8f923c83b2b067b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8aa4e1129d146d4a47f70f248771868" member_type: VOTER last_known_addr { host: "127.12.138.193" port: 38359 } } peers { permanent_uuid: "e6cbb5d02dcb40edb34ab5d41193496c" member_type: VOTER last_known_addr { host: "127.12.138.194" port: 33581 } } peers { permanent_uuid: "c0f254aed5f64d8b8f923c83b2b067b7" member_type: VOTER last_known_addr { host: "127.12.138.195" port: 42421 } }
I20260811 02:51:58.866356 15526 consensus_queue.cc:260] T e5f89aecee624e30b430ac9329c17d4a P c8aa4e1129d146d4a47f70f248771868 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8aa4e1129d146d4a47f70f248771868" member_type: VOTER last_known_addr { host: "127.12.138.193" port: 38359 } } peers { permanent_uuid: "e6cbb5d02dcb40edb34ab5d41193496c" member_type: VOTER last_known_addr { host: "127.12.138.194" port: 33581 } } peers { permanent_uuid: "c0f254aed5f64d8b8f923c83b2b067b7" member_type: VOTER last_known_addr { host: "127.12.138.195" port: 42421 } }
I20260811 02:51:58.866626 15526 ts_tablet_manager.cc:1428] T e5f89aecee624e30b430ac9329c17d4a P c8aa4e1129d146d4a47f70f248771868: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260811 02:51:58.866649 15525 ts_tablet_manager.cc:1428] T e5f89aecee624e30b430ac9329c17d4a P c0f254aed5f64d8b8f923c83b2b067b7: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260811 02:51:58.869925 15524 tablet_bootstrap.cc:492] T e5f89aecee624e30b430ac9329c17d4a P e6cbb5d02dcb40edb34ab5d41193496c: No bootstrap required, opened a new log
I20260811 02:51:58.870033 15524 ts_tablet_manager.cc:1397] T e5f89aecee624e30b430ac9329c17d4a P e6cbb5d02dcb40edb34ab5d41193496c: Time spent bootstrapping tablet: real 0.007s user 0.002s sys 0.000s
I20260811 02:51:58.870426 15524 raft_consensus.cc:348] T e5f89aecee624e30b430ac9329c17d4a P e6cbb5d02dcb40edb34ab5d41193496c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8aa4e1129d146d4a47f70f248771868" member_type: VOTER last_known_addr { host: "127.12.138.193" port: 38359 } } peers { permanent_uuid: "e6cbb5d02dcb40edb34ab5d41193496c" member_type: VOTER last_known_addr { host: "127.12.138.194" port: 33581 } } peers { permanent_uuid: "c0f254aed5f64d8b8f923c83b2b067b7" member_type: VOTER last_known_addr { host: "127.12.138.195" port: 42421 } }
I20260811 02:51:58.870553 15524 raft_consensus.cc:374] T e5f89aecee624e30b430ac9329c17d4a P e6cbb5d02dcb40edb34ab5d41193496c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 02:51:58.870596 15524 raft_consensus.cc:729] T e5f89aecee624e30b430ac9329c17d4a P e6cbb5d02dcb40edb34ab5d41193496c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e6cbb5d02dcb40edb34ab5d41193496c, State: Initialized, Role: FOLLOWER
I20260811 02:51:58.870760 15524 consensus_queue.cc:260] T e5f89aecee624e30b430ac9329c17d4a P e6cbb5d02dcb40edb34ab5d41193496c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8aa4e1129d146d4a47f70f248771868" member_type: VOTER last_known_addr { host: "127.12.138.193" port: 38359 } } peers { permanent_uuid: "e6cbb5d02dcb40edb34ab5d41193496c" member_type: VOTER last_known_addr { host: "127.12.138.194" port: 33581 } } peers { permanent_uuid: "c0f254aed5f64d8b8f923c83b2b067b7" member_type: VOTER last_known_addr { host: "127.12.138.195" port: 42421 } }
I20260811 02:51:58.871071 15524 ts_tablet_manager.cc:1428] T e5f89aecee624e30b430ac9329c17d4a P e6cbb5d02dcb40edb34ab5d41193496c: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260811 02:51:58.981340 15533 raft_consensus.cc:482] T e5f89aecee624e30b430ac9329c17d4a P c0f254aed5f64d8b8f923c83b2b067b7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 02:51:58.981515 15533 raft_consensus.cc:504] T e5f89aecee624e30b430ac9329c17d4a P c0f254aed5f64d8b8f923c83b2b067b7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8aa4e1129d146d4a47f70f248771868" member_type: VOTER last_known_addr { host: "127.12.138.193" port: 38359 } } peers { permanent_uuid: "e6cbb5d02dcb40edb34ab5d41193496c" member_type: VOTER last_known_addr { host: "127.12.138.194" port: 33581 } } peers { permanent_uuid: "c0f254aed5f64d8b8f923c83b2b067b7" member_type: VOTER last_known_addr { host: "127.12.138.195" port: 42421 } }
I20260811 02:51:58.982652 15533 leader_election.cc:288] T e5f89aecee624e30b430ac9329c17d4a P c0f254aed5f64d8b8f923c83b2b067b7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c8aa4e1129d146d4a47f70f248771868 (127.12.138.193:38359), e6cbb5d02dcb40edb34ab5d41193496c (127.12.138.194:33581)
I20260811 02:51:58.987259 15586 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:51:58.983444 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.138.194:33581 (local address 127.12.138.195:60305)
0811 02:51:58.983775 (+ 331us) negotiation.cc:107] Waiting for socket to connect
0811 02:51:58.983786 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0811 02:51:58.983907 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:51:58.984151 (+ 244us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:51:58.984154 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:51:58.984171 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 02:51:58.984291 (+ 120us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:51:58.984296 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:51:58.985026 (+ 730us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:51:58.985030 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:51:58.986074 (+ 1044us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:51:58.986084 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:51:58.986796 (+ 712us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:51:58.986800 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:51:58.986903 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:51:58.986932 (+ 29us) client_negotiation.cc:769] Sending connection context
0811 02:51:58.987011 (+ 79us) client_negotiation.cc:240] Negotiation successful
0811 02:51:58.987082 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":268,"mutex_wait_us":59,"thread_start_us":60,"threads_started":1}
I20260811 02:51:58.987270 15519 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:51:58.983539 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.138.195:60305 (local address 127.12.138.194:33581)
0811 02:51:58.983669 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0811 02:51:58.983674 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 02:51:58.983866 (+ 192us) server_negotiation.cc:407] Connection header received
0811 02:51:58.983996 (+ 130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:51:58.983999 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:51:58.984049 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:51:58.984135 (+ 86us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 02:51:58.984398 (+ 263us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:51:58.984905 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:51:58.986197 (+ 1292us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:51:58.986673 (+ 476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:51:58.986793 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:51:58.986844 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0811 02:51:58.987039 (+ 195us) server_negotiation.cc:299] Negotiation successful
0811 02:51:58.987113 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260811 02:51:58.987833 15522 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:51:58.983088 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.138.195:60793 (local address 127.12.138.193:38359)
0811 02:51:58.983250 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0811 02:51:58.983257 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 02:51:58.983423 (+ 166us) server_negotiation.cc:407] Connection header received
0811 02:51:58.983640 (+ 217us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:51:58.983644 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:51:58.983690 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:51:58.983739 (+ 49us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 02:51:58.984181 (+ 442us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:51:58.984966 (+ 785us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:51:58.986525 (+ 1559us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:51:58.987119 (+ 594us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:51:58.987240 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:51:58.987293 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0811 02:51:58.987610 (+ 317us) server_negotiation.cc:299] Negotiation successful
0811 02:51:58.987713 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260811 02:51:58.987916 15279 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e5f89aecee624e30b430ac9329c17d4a" candidate_uuid: "c0f254aed5f64d8b8f923c83b2b067b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e6cbb5d02dcb40edb34ab5d41193496c" is_pre_election: true
I20260811 02:51:58.988077 15279 raft_consensus.cc:2454] T e5f89aecee624e30b430ac9329c17d4a P e6cbb5d02dcb40edb34ab5d41193496c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c0f254aed5f64d8b8f923c83b2b067b7 in term 0.
I20260811 02:51:58.988233 15585 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:51:58.982997 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.138.193:38359 (local address 127.12.138.195:60793)
0811 02:51:58.983323 (+ 326us) negotiation.cc:107] Waiting for socket to connect
0811 02:51:58.983341 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 02:51:58.983510 (+ 169us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:51:58.983887 (+ 377us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:51:58.983889 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:51:58.983917 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 02:51:58.984042 (+ 125us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:51:58.984048 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:51:58.985123 (+ 1075us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:51:58.985126 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:51:58.986393 (+ 1267us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:51:58.986404 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:51:58.987361 (+ 957us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:51:58.987365 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:51:58.987464 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:51:58.987486 (+ 22us) client_negotiation.cc:769] Sending connection context
0811 02:51:58.987998 (+ 512us) client_negotiation.cc:240] Negotiation successful
0811 02:51:58.988081 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":241,"mutex_wait_us":25,"spinlock_wait_cycles":1792,"thread_start_us":96,"threads_started":1}
I20260811 02:51:58.988488 15363 leader_election.cc:302] T e5f89aecee624e30b430ac9329c17d4a P c0f254aed5f64d8b8f923c83b2b067b7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c0f254aed5f64d8b8f923c83b2b067b7, e6cbb5d02dcb40edb34ab5d41193496c; no voters:
I20260811 02:51:58.988814 15533 raft_consensus.cc:2781] T e5f89aecee624e30b430ac9329c17d4a P c0f254aed5f64d8b8f923c83b2b067b7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 02:51:58.988875 15533 raft_consensus.cc:482] T e5f89aecee624e30b430ac9329c17d4a P c0f254aed5f64d8b8f923c83b2b067b7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 02:51:58.988911 15533 raft_consensus.cc:3037] T e5f89aecee624e30b430ac9329c17d4a P c0f254aed5f64d8b8f923c83b2b067b7 [term 0 FOLLOWER]: Advancing to term 1
I20260811 02:51:58.989176 15115 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e5f89aecee624e30b430ac9329c17d4a" candidate_uuid: "c0f254aed5f64d8b8f923c83b2b067b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c8aa4e1129d146d4a47f70f248771868" is_pre_election: true
I20260811 02:51:58.989444 15115 raft_consensus.cc:2454] T e5f89aecee624e30b430ac9329c17d4a P c8aa4e1129d146d4a47f70f248771868 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c0f254aed5f64d8b8f923c83b2b067b7 in term 0.
I20260811 02:51:58.990274 15533 raft_consensus.cc:504] T e5f89aecee624e30b430ac9329c17d4a P c0f254aed5f64d8b8f923c83b2b067b7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8aa4e1129d146d4a47f70f248771868" member_type: VOTER last_known_addr { host: "127.12.138.193" port: 38359 } } peers { permanent_uuid: "e6cbb5d02dcb40edb34ab5d41193496c" member_type: VOTER last_known_addr { host: "127.12.138.194" port: 33581 } } peers { permanent_uuid: "c0f254aed5f64d8b8f923c83b2b067b7" member_type: VOTER last_known_addr { host: "127.12.138.195" port: 42421 } }
I20260811 02:51:58.990643 15533 leader_election.cc:288] T e5f89aecee624e30b430ac9329c17d4a P c0f254aed5f64d8b8f923c83b2b067b7 [CANDIDATE]: Term 1 election: Requested vote from peers c8aa4e1129d146d4a47f70f248771868 (127.12.138.193:38359), e6cbb5d02dcb40edb34ab5d41193496c (127.12.138.194:33581)
I20260811 02:51:58.991079 15115 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e5f89aecee624e30b430ac9329c17d4a" candidate_uuid: "c0f254aed5f64d8b8f923c83b2b067b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c8aa4e1129d146d4a47f70f248771868"
I20260811 02:51:58.991217 15115 raft_consensus.cc:3037] T e5f89aecee624e30b430ac9329c17d4a P c8aa4e1129d146d4a47f70f248771868 [term 0 FOLLOWER]: Advancing to term 1
I20260811 02:51:58.991226 15279 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e5f89aecee624e30b430ac9329c17d4a" candidate_uuid: "c0f254aed5f64d8b8f923c83b2b067b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e6cbb5d02dcb40edb34ab5d41193496c"
I20260811 02:51:58.991353 15279 raft_consensus.cc:3037] T e5f89aecee624e30b430ac9329c17d4a P e6cbb5d02dcb40edb34ab5d41193496c [term 0 FOLLOWER]: Advancing to term 1
I20260811 02:51:58.992486 15115 raft_consensus.cc:2454] T e5f89aecee624e30b430ac9329c17d4a P c8aa4e1129d146d4a47f70f248771868 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c0f254aed5f64d8b8f923c83b2b067b7 in term 1.
I20260811 02:51:58.992486 15279 raft_consensus.cc:2454] T e5f89aecee624e30b430ac9329c17d4a P e6cbb5d02dcb40edb34ab5d41193496c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c0f254aed5f64d8b8f923c83b2b067b7 in term 1.
I20260811 02:51:58.992779 15363 leader_election.cc:302] T e5f89aecee624e30b430ac9329c17d4a P c0f254aed5f64d8b8f923c83b2b067b7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c0f254aed5f64d8b8f923c83b2b067b7, e6cbb5d02dcb40edb34ab5d41193496c; no voters:
I20260811 02:51:58.992960 15533 raft_consensus.cc:2781] T e5f89aecee624e30b430ac9329c17d4a P c0f254aed5f64d8b8f923c83b2b067b7 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 02:51:58.993197 15533 raft_consensus.cc:686] T e5f89aecee624e30b430ac9329c17d4a P c0f254aed5f64d8b8f923c83b2b067b7 [term 1 LEADER]: Becoming Leader. State: Replica: c0f254aed5f64d8b8f923c83b2b067b7, State: Running, Role: LEADER
I20260811 02:51:58.993651 15533 consensus_queue.cc:237] T e5f89aecee624e30b430ac9329c17d4a P c0f254aed5f64d8b8f923c83b2b067b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8aa4e1129d146d4a47f70f248771868" member_type: VOTER last_known_addr { host: "127.12.138.193" port: 38359 } } peers { permanent_uuid: "e6cbb5d02dcb40edb34ab5d41193496c" member_type: VOTER last_known_addr { host: "127.12.138.194" port: 33581 } } peers { permanent_uuid: "c0f254aed5f64d8b8f923c83b2b067b7" member_type: VOTER last_known_addr { host: "127.12.138.195" port: 42421 } }
I20260811 02:51:58.996265 14928 catalog_manager.cc:5495] T e5f89aecee624e30b430ac9329c17d4a P c0f254aed5f64d8b8f923c83b2b067b7 reported cstate change: term changed from 0 to 1, leader changed from <none> to c0f254aed5f64d8b8f923c83b2b067b7 (127.12.138.195). New cstate: current_term: 1 leader_uuid: "c0f254aed5f64d8b8f923c83b2b067b7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8aa4e1129d146d4a47f70f248771868" member_type: VOTER last_known_addr { host: "127.12.138.193" port: 38359 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e6cbb5d02dcb40edb34ab5d41193496c" member_type: VOTER last_known_addr { host: "127.12.138.194" port: 33581 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c0f254aed5f64d8b8f923c83b2b067b7" member_type: VOTER last_known_addr { host: "127.12.138.195" port: 42421 } health_report { overall_health: HEALTHY } } }
I20260811 02:51:59.033213 15523 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:51:59.028983 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40922 (local address 127.12.138.195:42421)
0811 02:51:59.029097 (+ 114us) server_negotiation.cc:206] Beginning negotiation
0811 02:51:59.029100 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 02:51:59.029114 (+ 14us) server_negotiation.cc:407] Connection header received
0811 02:51:59.029343 (+ 229us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:51:59.029345 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:51:59.029398 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:51:59.029489 (+ 91us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 02:51:59.029778 (+ 289us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:51:59.030279 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:51:59.031463 (+ 1184us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:51:59.031759 (+ 296us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:51:59.031865 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:51:59.032226 (+ 361us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 02:51:59.032705 (+ 479us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 02:51:59.032801 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0811 02:51:59.032942 (+ 141us) server_negotiation.cc:299] Negotiation successful
0811 02:51:59.032995 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260811 02:51:59.046627 15522 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:51:59.042113 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.138.195:50207 (local address 127.12.138.193:38359)
0811 02:51:59.042286 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0811 02:51:59.042291 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 02:51:59.042309 (+ 18us) server_negotiation.cc:407] Connection header received
0811 02:51:59.042370 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:51:59.042373 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:51:59.042428 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:51:59.042529 (+ 101us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 02:51:59.043314 (+ 785us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:51:59.044080 (+ 766us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:51:59.045128 (+ 1048us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:51:59.045435 (+ 307us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:51:59.045594 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:51:59.046027 (+ 433us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 02:51:59.046111 (+ 84us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 02:51:59.046250 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0811 02:51:59.046402 (+ 152us) server_negotiation.cc:299] Negotiation successful
0811 02:51:59.046462 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260811 02:51:59.046667 15591 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:51:59.041720 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.138.193:38359 (local address 127.12.138.195:50207)
0811 02:51:59.042066 (+ 346us) negotiation.cc:107] Waiting for socket to connect
0811 02:51:59.042083 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 02:51:59.042196 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:51:59.042581 (+ 385us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:51:59.042584 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:51:59.042593 (+ 9us) client_negotiation.cc:189] Negotiated authn=TOKEN
0811 02:51:59.043164 (+ 571us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:51:59.043172 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:51:59.044230 (+ 1058us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:51:59.044233 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:51:59.044966 (+ 733us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:51:59.044976 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:51:59.045584 (+ 608us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:51:59.045588 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:51:59.045793 (+ 205us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:51:59.045850 (+ 57us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0811 02:51:59.046268 (+ 418us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0811 02:51:59.046279 (+ 11us) client_negotiation.cc:769] Sending connection context
0811 02:51:59.046380 (+ 101us) client_negotiation.cc:240] Negotiation successful
0811 02:51:59.046475 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":227,"thread_start_us":134,"threads_started":1}
I20260811 02:51:59.054553 15115 raft_consensus.cc:1261] T e5f89aecee624e30b430ac9329c17d4a P c8aa4e1129d146d4a47f70f248771868 [term 1 FOLLOWER]: Refusing update from remote peer c0f254aed5f64d8b8f923c83b2b067b7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 02:51:59.055039 15279 raft_consensus.cc:1261] T e5f89aecee624e30b430ac9329c17d4a P e6cbb5d02dcb40edb34ab5d41193496c [term 1 FOLLOWER]: Refusing update from remote peer c0f254aed5f64d8b8f923c83b2b067b7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 02:51:59.055277 15587 consensus_queue.cc:1035] T e5f89aecee624e30b430ac9329c17d4a P c0f254aed5f64d8b8f923c83b2b067b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c8aa4e1129d146d4a47f70f248771868" member_type: VOTER last_known_addr { host: "127.12.138.193" port: 38359 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 02:51:59.055550 15533 consensus_queue.cc:1035] T e5f89aecee624e30b430ac9329c17d4a P c0f254aed5f64d8b8f923c83b2b067b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e6cbb5d02dcb40edb34ab5d41193496c" member_type: VOTER last_known_addr { host: "127.12.138.194" port: 33581 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 02:51:59.077452 15566 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:51:59.073445 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.138.195:42421 (local address 127.12.138.193:35897)
0811 02:51:59.073643 (+ 198us) negotiation.cc:107] Waiting for socket to connect
0811 02:51:59.073660 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 02:51:59.073744 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:51:59.074050 (+ 306us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:51:59.074053 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:51:59.074060 (+ 7us) client_negotiation.cc:189] Negotiated authn=TOKEN
0811 02:51:59.074164 (+ 104us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:51:59.074170 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:51:59.075013 (+ 843us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:51:59.075018 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:51:59.076085 (+ 1067us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:51:59.076099 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:51:59.076665 (+ 566us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:51:59.076669 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:51:59.076776 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:51:59.076814 (+ 38us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0811 02:51:59.077090 (+ 276us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0811 02:51:59.077097 (+ 7us) client_negotiation.cc:769] Sending connection context
0811 02:51:59.077207 (+ 110us) client_negotiation.cc:240] Negotiation successful
0811 02:51:59.077300 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":97}
I20260811 02:51:59.077409 15523 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:51:59.073659 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.138.193:35897 (local address 127.12.138.195:42421)
0811 02:51:59.073819 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0811 02:51:59.073826 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 02:51:59.073840 (+ 14us) server_negotiation.cc:407] Connection header received
0811 02:51:59.073879 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:51:59.073882 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:51:59.073931 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:51:59.074039 (+ 108us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 02:51:59.074276 (+ 237us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:51:59.074848 (+ 572us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:51:59.076206 (+ 1358us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:51:59.076539 (+ 333us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:51:59.076646 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:51:59.076936 (+ 290us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 02:51:59.077005 (+ 69us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 02:51:59.077054 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0811 02:51:59.077215 (+ 161us) server_negotiation.cc:299] Negotiation successful
0811 02:51:59.077275 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
W20260811 02:51:59.078244 15555 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260811 02:52:00.647504 14928 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50218:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 02:52:00.647688 14928 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50218:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 02:52:00.650408 14928 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 60ef762b8fc246488dfb261f0b1e8420: Sending DeleteTablet for 3 replicas of tablet e5f89aecee624e30b430ac9329c17d4a
I20260811 02:52:00.651063 15090 tablet_service.cc:1514] Processing DeleteTablet for tablet e5f89aecee624e30b430ac9329c17d4a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 02:52:00 UTC) from {username='kudu', principal='kudu/127.12.138.254@KRBTEST.COM'} at 127.0.0.1:37042
I20260811 02:52:00.651142 15258 tablet_service.cc:1514] Processing DeleteTablet for tablet e5f89aecee624e30b430ac9329c17d4a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 02:52:00 UTC) from {username='kudu', principal='kudu/127.12.138.254@KRBTEST.COM'} at 127.0.0.1:53890
I20260811 02:52:00.651314 15421 tablet_service.cc:1514] Processing DeleteTablet for tablet e5f89aecee624e30b430ac9329c17d4a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 02:52:00 UTC) from {username='kudu', principal='kudu/127.12.138.254@KRBTEST.COM'} at 127.0.0.1:40906
I20260811 02:52:00.651539 15624 tablet_replica.cc:330] T e5f89aecee624e30b430ac9329c17d4a P c8aa4e1129d146d4a47f70f248771868: stopping tablet replica
I20260811 02:52:00.651638 15625 tablet_replica.cc:330] T e5f89aecee624e30b430ac9329c17d4a P e6cbb5d02dcb40edb34ab5d41193496c: stopping tablet replica
I20260811 02:52:00.651686 15624 raft_consensus.cc:2229] T e5f89aecee624e30b430ac9329c17d4a P c8aa4e1129d146d4a47f70f248771868 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 02:52:00.651815 15626 tablet_replica.cc:330] T e5f89aecee624e30b430ac9329c17d4a P c0f254aed5f64d8b8f923c83b2b067b7: stopping tablet replica
I20260811 02:52:00.651860 15624 raft_consensus.cc:2258] T e5f89aecee624e30b430ac9329c17d4a P c8aa4e1129d146d4a47f70f248771868 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 02:52:00.652297 15626 raft_consensus.cc:2229] T e5f89aecee624e30b430ac9329c17d4a P c0f254aed5f64d8b8f923c83b2b067b7 [term 1 LEADER]: Raft consensus shutting down.
I20260811 02:52:00.652434 15625 raft_consensus.cc:2229] T e5f89aecee624e30b430ac9329c17d4a P e6cbb5d02dcb40edb34ab5d41193496c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 02:52:00.652626 15625 raft_consensus.cc:2258] T e5f89aecee624e30b430ac9329c17d4a P e6cbb5d02dcb40edb34ab5d41193496c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 02:52:00.653358 15626 raft_consensus.cc:2258] T e5f89aecee624e30b430ac9329c17d4a P c0f254aed5f64d8b8f923c83b2b067b7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 02:52:00.654167 15624 ts_tablet_manager.cc:1905] T e5f89aecee624e30b430ac9329c17d4a P c8aa4e1129d146d4a47f70f248771868: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 02:52:00.655283 15625 ts_tablet_manager.cc:1905] T e5f89aecee624e30b430ac9329c17d4a P e6cbb5d02dcb40edb34ab5d41193496c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 02:52:00.656080 15626 ts_tablet_manager.cc:1905] T e5f89aecee624e30b430ac9329c17d4a P c0f254aed5f64d8b8f923c83b2b067b7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 02:52:00.657840 15625 ts_tablet_manager.cc:1918] T e5f89aecee624e30b430ac9329c17d4a P e6cbb5d02dcb40edb34ab5d41193496c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260811 02:52:00.657974 15625 log.cc:1196] T e5f89aecee624e30b430ac9329c17d4a P e6cbb5d02dcb40edb34ab5d41193496c: Deleting WAL directory at /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786416707705494-12843-0/minicluster-data/ts-1/wal/wals/e5f89aecee624e30b430ac9329c17d4a
I20260811 02:52:00.658303 15625 ts_tablet_manager.cc:1939] T e5f89aecee624e30b430ac9329c17d4a P e6cbb5d02dcb40edb34ab5d41193496c: Deleting consensus metadata
I20260811 02:52:00.659107 14918 catalog_manager.cc:4841] TS e6cbb5d02dcb40edb34ab5d41193496c (127.12.138.194:33581): tablet e5f89aecee624e30b430ac9329c17d4a (table test-table [id=a19b76a9f24643c787f2433fb034ddb8]) successfully deleted
I20260811 02:52:00.659528 15624 ts_tablet_manager.cc:1918] T e5f89aecee624e30b430ac9329c17d4a P c8aa4e1129d146d4a47f70f248771868: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260811 02:52:00.659632 15624 log.cc:1196] T e5f89aecee624e30b430ac9329c17d4a P c8aa4e1129d146d4a47f70f248771868: Deleting WAL directory at /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786416707705494-12843-0/minicluster-data/ts-0/wal/wals/e5f89aecee624e30b430ac9329c17d4a
I20260811 02:52:00.659734 15626 ts_tablet_manager.cc:1918] T e5f89aecee624e30b430ac9329c17d4a P c0f254aed5f64d8b8f923c83b2b067b7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260811 02:52:00.659906 15626 log.cc:1196] T e5f89aecee624e30b430ac9329c17d4a P c0f254aed5f64d8b8f923c83b2b067b7: Deleting WAL directory at /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786416707705494-12843-0/minicluster-data/ts-2/wal/wals/e5f89aecee624e30b430ac9329c17d4a
I20260811 02:52:00.660012 15624 ts_tablet_manager.cc:1939] T e5f89aecee624e30b430ac9329c17d4a P c8aa4e1129d146d4a47f70f248771868: Deleting consensus metadata
I20260811 02:52:00.660257 15626 ts_tablet_manager.cc:1939] T e5f89aecee624e30b430ac9329c17d4a P c0f254aed5f64d8b8f923c83b2b067b7: Deleting consensus metadata
I20260811 02:52:00.660900 14918 catalog_manager.cc:4841] TS c8aa4e1129d146d4a47f70f248771868 (127.12.138.193:38359): tablet e5f89aecee624e30b430ac9329c17d4a (table test-table [id=a19b76a9f24643c787f2433fb034ddb8]) successfully deleted
I20260811 02:52:00.660979 14919 catalog_manager.cc:4841] TS c0f254aed5f64d8b8f923c83b2b067b7 (127.12.138.195:42421): tablet e5f89aecee624e30b430ac9329c17d4a (table test-table [id=a19b76a9f24643c787f2433fb034ddb8]) successfully deleted
Aug 11 02:52:00 dist-test-slave-9gcw krb5kdc[14879](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416720, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 11 02:52:00 dist-test-slave-9gcw krb5kdc[14879](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416720, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.12.138.254@KRBTEST.COM
I20260811 02:52:00.687407 15636 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:52:00.674158 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50234 (local address 127.12.138.254:41515)
0811 02:52:00.674532 (+ 374us) server_negotiation.cc:206] Beginning negotiation
0811 02:52:00.674541 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 02:52:00.674559 (+ 18us) server_negotiation.cc:407] Connection header received
0811 02:52:00.674690 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:52:00.674693 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:52:00.674748 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:52:00.674887 (+ 139us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:52:00.675685 (+ 798us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:52:00.676265 (+ 580us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:52:00.677200 (+ 935us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:52:00.677470 (+ 270us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:52:00.677583 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:52:00.683865 (+ 6282us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:52:00.683868 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:52:00.683870 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:52:00.683899 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:52:00.685671 (+ 1772us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:52:00.686171 (+ 500us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:52:00.686174 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:52:00.686175 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:52:00.686256 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:52:00.686514 (+ 258us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:52:00.686516 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:52:00.686517 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:52:00.686722 (+ 205us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:52:00.686850 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0811 02:52:00.687149 (+ 299us) server_negotiation.cc:299] Negotiation successful
0811 02:52:00.687265 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":283,"thread_start_us":122,"threads_started":1}
I20260811 02:52:00.690500 14928 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:50234:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 02:52:00.690970 14928 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 02:52:00.695894 15090 tablet_service.cc:1467] Processing CreateTablet for tablet 55f73b1e296446a192974d93f0a7ef37 (DEFAULT_TABLE table=test-table [id=8cea614cd44d495b9377a1c9bdb29895]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 02:52:00.696010 15258 tablet_service.cc:1467] Processing CreateTablet for tablet 55f73b1e296446a192974d93f0a7ef37 (DEFAULT_TABLE table=test-table [id=8cea614cd44d495b9377a1c9bdb29895]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 02:52:00.696168 15421 tablet_service.cc:1467] Processing CreateTablet for tablet 55f73b1e296446a192974d93f0a7ef37 (DEFAULT_TABLE table=test-table [id=8cea614cd44d495b9377a1c9bdb29895]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 02:52:00.696292 15090 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 55f73b1e296446a192974d93f0a7ef37. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 02:52:00.696368 15258 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 55f73b1e296446a192974d93f0a7ef37. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 02:52:00.696444 15421 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 55f73b1e296446a192974d93f0a7ef37. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 02:52:00.698860 15637 tablet_bootstrap.cc:492] T 55f73b1e296446a192974d93f0a7ef37 P e6cbb5d02dcb40edb34ab5d41193496c: Bootstrap starting.
I20260811 02:52:00.699450 15638 tablet_bootstrap.cc:492] T 55f73b1e296446a192974d93f0a7ef37 P c8aa4e1129d146d4a47f70f248771868: Bootstrap starting.
I20260811 02:52:00.699839 15637 tablet_bootstrap.cc:654] T 55f73b1e296446a192974d93f0a7ef37 P e6cbb5d02dcb40edb34ab5d41193496c: Neither blocks nor log segments found. Creating new log.
I20260811 02:52:00.700237 15638 tablet_bootstrap.cc:654] T 55f73b1e296446a192974d93f0a7ef37 P c8aa4e1129d146d4a47f70f248771868: Neither blocks nor log segments found. Creating new log.
I20260811 02:52:00.700876 15640 tablet_bootstrap.cc:492] T 55f73b1e296446a192974d93f0a7ef37 P c0f254aed5f64d8b8f923c83b2b067b7: Bootstrap starting.
I20260811 02:52:00.701068 15637 tablet_bootstrap.cc:492] T 55f73b1e296446a192974d93f0a7ef37 P e6cbb5d02dcb40edb34ab5d41193496c: No bootstrap required, opened a new log
I20260811 02:52:00.701210 15637 ts_tablet_manager.cc:1397] T 55f73b1e296446a192974d93f0a7ef37 P e6cbb5d02dcb40edb34ab5d41193496c: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260811 02:52:00.701665 15637 raft_consensus.cc:348] T 55f73b1e296446a192974d93f0a7ef37 P e6cbb5d02dcb40edb34ab5d41193496c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8aa4e1129d146d4a47f70f248771868" member_type: VOTER last_known_addr { host: "127.12.138.193" port: 38359 } } peers { permanent_uuid: "e6cbb5d02dcb40edb34ab5d41193496c" member_type: VOTER last_known_addr { host: "127.12.138.194" port: 33581 } } peers { permanent_uuid: "c0f254aed5f64d8b8f923c83b2b067b7" member_type: VOTER last_known_addr { host: "127.12.138.195" port: 42421 } }
I20260811 02:52:00.701821 15637 raft_consensus.cc:374] T 55f73b1e296446a192974d93f0a7ef37 P e6cbb5d02dcb40edb34ab5d41193496c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 02:52:00.701889 15637 raft_consensus.cc:729] T 55f73b1e296446a192974d93f0a7ef37 P e6cbb5d02dcb40edb34ab5d41193496c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e6cbb5d02dcb40edb34ab5d41193496c, State: Initialized, Role: FOLLOWER
I20260811 02:52:00.701982 15640 tablet_bootstrap.cc:654] T 55f73b1e296446a192974d93f0a7ef37 P c0f254aed5f64d8b8f923c83b2b067b7: Neither blocks nor log segments found. Creating new log.
I20260811 02:52:00.702054 15637 consensus_queue.cc:260] T 55f73b1e296446a192974d93f0a7ef37 P e6cbb5d02dcb40edb34ab5d41193496c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8aa4e1129d146d4a47f70f248771868" member_type: VOTER last_known_addr { host: "127.12.138.193" port: 38359 } } peers { permanent_uuid: "e6cbb5d02dcb40edb34ab5d41193496c" member_type: VOTER last_known_addr { host: "127.12.138.194" port: 33581 } } peers { permanent_uuid: "c0f254aed5f64d8b8f923c83b2b067b7" member_type: VOTER last_known_addr { host: "127.12.138.195" port: 42421 } }
I20260811 02:52:00.702464 15638 tablet_bootstrap.cc:492] T 55f73b1e296446a192974d93f0a7ef37 P c8aa4e1129d146d4a47f70f248771868: No bootstrap required, opened a new log
I20260811 02:52:00.702538 15638 ts_tablet_manager.cc:1397] T 55f73b1e296446a192974d93f0a7ef37 P c8aa4e1129d146d4a47f70f248771868: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260811 02:52:00.702879 15637 ts_tablet_manager.cc:1428] T 55f73b1e296446a192974d93f0a7ef37 P e6cbb5d02dcb40edb34ab5d41193496c: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20260811 02:52:00.702960 15638 raft_consensus.cc:348] T 55f73b1e296446a192974d93f0a7ef37 P c8aa4e1129d146d4a47f70f248771868 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8aa4e1129d146d4a47f70f248771868" member_type: VOTER last_known_addr { host: "127.12.138.193" port: 38359 } } peers { permanent_uuid: "e6cbb5d02dcb40edb34ab5d41193496c" member_type: VOTER last_known_addr { host: "127.12.138.194" port: 33581 } } peers { permanent_uuid: "c0f254aed5f64d8b8f923c83b2b067b7" member_type: VOTER last_known_addr { host: "127.12.138.195" port: 42421 } }
I20260811 02:52:00.703109 15638 raft_consensus.cc:374] T 55f73b1e296446a192974d93f0a7ef37 P c8aa4e1129d146d4a47f70f248771868 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 02:52:00.703158 15638 raft_consensus.cc:729] T 55f73b1e296446a192974d93f0a7ef37 P c8aa4e1129d146d4a47f70f248771868 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c8aa4e1129d146d4a47f70f248771868, State: Initialized, Role: FOLLOWER
I20260811 02:52:00.703308 15638 consensus_queue.cc:260] T 55f73b1e296446a192974d93f0a7ef37 P c8aa4e1129d146d4a47f70f248771868 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8aa4e1129d146d4a47f70f248771868" member_type: VOTER last_known_addr { host: "127.12.138.193" port: 38359 } } peers { permanent_uuid: "e6cbb5d02dcb40edb34ab5d41193496c" member_type: VOTER last_known_addr { host: "127.12.138.194" port: 33581 } } peers { permanent_uuid: "c0f254aed5f64d8b8f923c83b2b067b7" member_type: VOTER last_known_addr { host: "127.12.138.195" port: 42421 } }
I20260811 02:52:00.703577 15638 ts_tablet_manager.cc:1428] T 55f73b1e296446a192974d93f0a7ef37 P c8aa4e1129d146d4a47f70f248771868: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260811 02:52:00.705686 15640 tablet_bootstrap.cc:492] T 55f73b1e296446a192974d93f0a7ef37 P c0f254aed5f64d8b8f923c83b2b067b7: No bootstrap required, opened a new log
I20260811 02:52:00.705821 15640 ts_tablet_manager.cc:1397] T 55f73b1e296446a192974d93f0a7ef37 P c0f254aed5f64d8b8f923c83b2b067b7: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20260811 02:52:00.706168 15640 raft_consensus.cc:348] T 55f73b1e296446a192974d93f0a7ef37 P c0f254aed5f64d8b8f923c83b2b067b7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8aa4e1129d146d4a47f70f248771868" member_type: VOTER last_known_addr { host: "127.12.138.193" port: 38359 } } peers { permanent_uuid: "e6cbb5d02dcb40edb34ab5d41193496c" member_type: VOTER last_known_addr { host: "127.12.138.194" port: 33581 } } peers { permanent_uuid: "c0f254aed5f64d8b8f923c83b2b067b7" member_type: VOTER last_known_addr { host: "127.12.138.195" port: 42421 } }
I20260811 02:52:00.706264 15640 raft_consensus.cc:374] T 55f73b1e296446a192974d93f0a7ef37 P c0f254aed5f64d8b8f923c83b2b067b7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 02:52:00.706292 15640 raft_consensus.cc:729] T 55f73b1e296446a192974d93f0a7ef37 P c0f254aed5f64d8b8f923c83b2b067b7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c0f254aed5f64d8b8f923c83b2b067b7, State: Initialized, Role: FOLLOWER
I20260811 02:52:00.706447 15640 consensus_queue.cc:260] T 55f73b1e296446a192974d93f0a7ef37 P c0f254aed5f64d8b8f923c83b2b067b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8aa4e1129d146d4a47f70f248771868" member_type: VOTER last_known_addr { host: "127.12.138.193" port: 38359 } } peers { permanent_uuid: "e6cbb5d02dcb40edb34ab5d41193496c" member_type: VOTER last_known_addr { host: "127.12.138.194" port: 33581 } } peers { permanent_uuid: "c0f254aed5f64d8b8f923c83b2b067b7" member_type: VOTER last_known_addr { host: "127.12.138.195" port: 42421 } }
I20260811 02:52:00.706773 15640 ts_tablet_manager.cc:1428] T 55f73b1e296446a192974d93f0a7ef37 P c0f254aed5f64d8b8f923c83b2b067b7: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260811 02:52:00.888571 15643 raft_consensus.cc:482] T 55f73b1e296446a192974d93f0a7ef37 P e6cbb5d02dcb40edb34ab5d41193496c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 02:52:00.888725 15643 raft_consensus.cc:504] T 55f73b1e296446a192974d93f0a7ef37 P e6cbb5d02dcb40edb34ab5d41193496c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8aa4e1129d146d4a47f70f248771868" member_type: VOTER last_known_addr { host: "127.12.138.193" port: 38359 } } peers { permanent_uuid: "e6cbb5d02dcb40edb34ab5d41193496c" member_type: VOTER last_known_addr { host: "127.12.138.194" port: 33581 } } peers { permanent_uuid: "c0f254aed5f64d8b8f923c83b2b067b7" member_type: VOTER last_known_addr { host: "127.12.138.195" port: 42421 } }
I20260811 02:52:00.890098 15643 leader_election.cc:288] T 55f73b1e296446a192974d93f0a7ef37 P e6cbb5d02dcb40edb34ab5d41193496c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c8aa4e1129d146d4a47f70f248771868 (127.12.138.193:38359), c0f254aed5f64d8b8f923c83b2b067b7 (127.12.138.195:42421)
I20260811 02:52:00.894567 15647 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:52:00.890283 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.138.193:38359 (local address 127.12.138.194:51403)
0811 02:52:00.890633 (+ 350us) negotiation.cc:107] Waiting for socket to connect
0811 02:52:00.890649 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 02:52:00.890764 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:52:00.891098 (+ 334us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:52:00.891101 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:52:00.891120 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 02:52:00.891276 (+ 156us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:52:00.891284 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:52:00.892343 (+ 1059us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:52:00.892346 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:52:00.893240 (+ 894us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:52:00.893249 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:52:00.894004 (+ 755us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:52:00.894007 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:52:00.894176 (+ 169us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:52:00.894196 (+ 20us) client_negotiation.cc:769] Sending connection context
0811 02:52:00.894260 (+ 64us) client_negotiation.cc:240] Negotiation successful
0811 02:52:00.894362 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":272,"mutex_wait_us":95,"thread_start_us":144,"threads_started":1}
I20260811 02:52:00.894559 15650 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:52:00.890663 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.138.194:44237 (local address 127.12.138.195:42421)
0811 02:52:00.890991 (+ 328us) server_negotiation.cc:206] Beginning negotiation
0811 02:52:00.891003 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 02:52:00.891018 (+ 15us) server_negotiation.cc:407] Connection header received
0811 02:52:00.891098 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:52:00.891101 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:52:00.891145 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:52:00.891241 (+ 96us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 02:52:00.891630 (+ 389us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:52:00.892136 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:52:00.893454 (+ 1318us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:52:00.893871 (+ 417us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:52:00.894009 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:52:00.894079 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0811 02:52:00.894357 (+ 278us) server_negotiation.cc:299] Negotiation successful
0811 02:52:00.894444 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":261,"thread_start_us":102,"threads_started":1}
I20260811 02:52:00.894573 15649 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:52:00.890446 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.138.194:51403 (local address 127.12.138.193:38359)
0811 02:52:00.890800 (+ 354us) server_negotiation.cc:206] Beginning negotiation
0811 02:52:00.890812 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 02:52:00.890841 (+ 29us) server_negotiation.cc:407] Connection header received
0811 02:52:00.890906 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:52:00.890909 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:52:00.890967 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:52:00.891056 (+ 89us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 02:52:00.891440 (+ 384us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:52:00.892172 (+ 732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:52:00.893373 (+ 1201us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:52:00.893871 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:52:00.894020 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:52:00.894090 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0811 02:52:00.894324 (+ 234us) server_negotiation.cc:299] Negotiation successful
0811 02:52:00.894411 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":282,"thread_start_us":128,"threads_started":1}
I20260811 02:52:00.894594 15648 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:52:00.890426 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.138.195:42421 (local address 127.12.138.194:44237)
0811 02:52:00.890796 (+ 370us) negotiation.cc:107] Waiting for socket to connect
0811 02:52:00.890812 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 02:52:00.890914 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:52:00.891291 (+ 377us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:52:00.891294 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:52:00.891312 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 02:52:00.891493 (+ 181us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:52:00.891502 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:52:00.892271 (+ 769us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:52:00.892276 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:52:00.893313 (+ 1037us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:52:00.893329 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:52:00.894004 (+ 675us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:52:00.894008 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:52:00.894181 (+ 173us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:52:00.894202 (+ 21us) client_negotiation.cc:769] Sending connection context
0811 02:52:00.894313 (+ 111us) client_negotiation.cc:240] Negotiation successful
0811 02:52:00.894394 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":306,"mutex_wait_us":27,"thread_start_us":116,"threads_started":1}
I20260811 02:52:00.895174 15115 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "55f73b1e296446a192974d93f0a7ef37" candidate_uuid: "e6cbb5d02dcb40edb34ab5d41193496c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c8aa4e1129d146d4a47f70f248771868" is_pre_election: true
I20260811 02:52:00.895190 15438 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "55f73b1e296446a192974d93f0a7ef37" candidate_uuid: "e6cbb5d02dcb40edb34ab5d41193496c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c0f254aed5f64d8b8f923c83b2b067b7" is_pre_election: true
I20260811 02:52:00.895319 15115 raft_consensus.cc:2454] T 55f73b1e296446a192974d93f0a7ef37 P c8aa4e1129d146d4a47f70f248771868 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e6cbb5d02dcb40edb34ab5d41193496c in term 0.
I20260811 02:52:00.895394 15438 raft_consensus.cc:2454] T 55f73b1e296446a192974d93f0a7ef37 P c0f254aed5f64d8b8f923c83b2b067b7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e6cbb5d02dcb40edb34ab5d41193496c in term 0.
I20260811 02:52:00.895810 15198 leader_election.cc:302] T 55f73b1e296446a192974d93f0a7ef37 P e6cbb5d02dcb40edb34ab5d41193496c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: c0f254aed5f64d8b8f923c83b2b067b7, c8aa4e1129d146d4a47f70f248771868, e6cbb5d02dcb40edb34ab5d41193496c; no voters:
I20260811 02:52:00.896137 15643 raft_consensus.cc:2781] T 55f73b1e296446a192974d93f0a7ef37 P e6cbb5d02dcb40edb34ab5d41193496c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 02:52:00.896205 15643 raft_consensus.cc:482] T 55f73b1e296446a192974d93f0a7ef37 P e6cbb5d02dcb40edb34ab5d41193496c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 02:52:00.896234 15643 raft_consensus.cc:3037] T 55f73b1e296446a192974d93f0a7ef37 P e6cbb5d02dcb40edb34ab5d41193496c [term 0 FOLLOWER]: Advancing to term 1
I20260811 02:52:00.897353 15643 raft_consensus.cc:504] T 55f73b1e296446a192974d93f0a7ef37 P e6cbb5d02dcb40edb34ab5d41193496c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8aa4e1129d146d4a47f70f248771868" member_type: VOTER last_known_addr { host: "127.12.138.193" port: 38359 } } peers { permanent_uuid: "e6cbb5d02dcb40edb34ab5d41193496c" member_type: VOTER last_known_addr { host: "127.12.138.194" port: 33581 } } peers { permanent_uuid: "c0f254aed5f64d8b8f923c83b2b067b7" member_type: VOTER last_known_addr { host: "127.12.138.195" port: 42421 } }
I20260811 02:52:00.897698 15643 leader_election.cc:288] T 55f73b1e296446a192974d93f0a7ef37 P e6cbb5d02dcb40edb34ab5d41193496c [CANDIDATE]: Term 1 election: Requested vote from peers c8aa4e1129d146d4a47f70f248771868 (127.12.138.193:38359), c0f254aed5f64d8b8f923c83b2b067b7 (127.12.138.195:42421)
I20260811 02:52:00.898052 15438 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "55f73b1e296446a192974d93f0a7ef37" candidate_uuid: "e6cbb5d02dcb40edb34ab5d41193496c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c0f254aed5f64d8b8f923c83b2b067b7"
I20260811 02:52:00.898108 15115 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "55f73b1e296446a192974d93f0a7ef37" candidate_uuid: "e6cbb5d02dcb40edb34ab5d41193496c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c8aa4e1129d146d4a47f70f248771868"
I20260811 02:52:00.898191 15438 raft_consensus.cc:3037] T 55f73b1e296446a192974d93f0a7ef37 P c0f254aed5f64d8b8f923c83b2b067b7 [term 0 FOLLOWER]: Advancing to term 1
I20260811 02:52:00.898272 15115 raft_consensus.cc:3037] T 55f73b1e296446a192974d93f0a7ef37 P c8aa4e1129d146d4a47f70f248771868 [term 0 FOLLOWER]: Advancing to term 1
I20260811 02:52:00.899155 15438 raft_consensus.cc:2454] T 55f73b1e296446a192974d93f0a7ef37 P c0f254aed5f64d8b8f923c83b2b067b7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e6cbb5d02dcb40edb34ab5d41193496c in term 1.
I20260811 02:52:00.899459 15115 raft_consensus.cc:2454] T 55f73b1e296446a192974d93f0a7ef37 P c8aa4e1129d146d4a47f70f248771868 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e6cbb5d02dcb40edb34ab5d41193496c in term 1.
I20260811 02:52:00.899477 15199 leader_election.cc:302] T 55f73b1e296446a192974d93f0a7ef37 P e6cbb5d02dcb40edb34ab5d41193496c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c0f254aed5f64d8b8f923c83b2b067b7, e6cbb5d02dcb40edb34ab5d41193496c; no voters:
I20260811 02:52:00.899657 15643 raft_consensus.cc:2781] T 55f73b1e296446a192974d93f0a7ef37 P e6cbb5d02dcb40edb34ab5d41193496c [term 1 FOLLOWER]: Leader election won for term 1
I20260811 02:52:00.899879 15643 raft_consensus.cc:686] T 55f73b1e296446a192974d93f0a7ef37 P e6cbb5d02dcb40edb34ab5d41193496c [term 1 LEADER]: Becoming Leader. State: Replica: e6cbb5d02dcb40edb34ab5d41193496c, State: Running, Role: LEADER
I20260811 02:52:00.900142 15643 consensus_queue.cc:237] T 55f73b1e296446a192974d93f0a7ef37 P e6cbb5d02dcb40edb34ab5d41193496c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8aa4e1129d146d4a47f70f248771868" member_type: VOTER last_known_addr { host: "127.12.138.193" port: 38359 } } peers { permanent_uuid: "e6cbb5d02dcb40edb34ab5d41193496c" member_type: VOTER last_known_addr { host: "127.12.138.194" port: 33581 } } peers { permanent_uuid: "c0f254aed5f64d8b8f923c83b2b067b7" member_type: VOTER last_known_addr { host: "127.12.138.195" port: 42421 } }
I20260811 02:52:00.902479 14928 catalog_manager.cc:5495] T 55f73b1e296446a192974d93f0a7ef37 P e6cbb5d02dcb40edb34ab5d41193496c reported cstate change: term changed from 0 to 1, leader changed from <none> to e6cbb5d02dcb40edb34ab5d41193496c (127.12.138.194). New cstate: current_term: 1 leader_uuid: "e6cbb5d02dcb40edb34ab5d41193496c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8aa4e1129d146d4a47f70f248771868" member_type: VOTER last_known_addr { host: "127.12.138.193" port: 38359 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e6cbb5d02dcb40edb34ab5d41193496c" member_type: VOTER last_known_addr { host: "127.12.138.194" port: 33581 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c0f254aed5f64d8b8f923c83b2b067b7" member_type: VOTER last_known_addr { host: "127.12.138.195" port: 42421 } health_report { overall_health: UNKNOWN } } }
I20260811 02:52:00.947229 15656 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:52:00.942445 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53902 (local address 127.12.138.194:33581)
0811 02:52:00.942731 (+ 286us) server_negotiation.cc:206] Beginning negotiation
0811 02:52:00.942737 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 02:52:00.942750 (+ 13us) server_negotiation.cc:407] Connection header received
0811 02:52:00.942842 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:52:00.942844 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:52:00.942898 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:52:00.943005 (+ 107us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 02:52:00.943282 (+ 277us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:52:00.943915 (+ 633us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:52:00.945015 (+ 1100us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:52:00.945526 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:52:00.945613 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:52:00.946021 (+ 408us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 02:52:00.946670 (+ 649us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 02:52:00.946796 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0811 02:52:00.946952 (+ 156us) server_negotiation.cc:299] Negotiation successful
0811 02:52:00.947018 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":223,"thread_start_us":140,"threads_started":1}
I20260811 02:52:00.957686 15662 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:52:00.954126 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.138.193:38359 (local address 127.12.138.194:44453)
0811 02:52:00.954482 (+ 356us) negotiation.cc:107] Waiting for socket to connect
0811 02:52:00.954498 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 02:52:00.954599 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:52:00.954913 (+ 314us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:52:00.954916 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:52:00.954922 (+ 6us) client_negotiation.cc:189] Negotiated authn=TOKEN
0811 02:52:00.955021 (+ 99us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:52:00.955026 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:52:00.955767 (+ 741us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:52:00.955769 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:52:00.956406 (+ 637us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:52:00.956415 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:52:00.956906 (+ 491us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:52:00.956909 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:52:00.956997 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:52:00.957041 (+ 44us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0811 02:52:00.957412 (+ 371us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0811 02:52:00.957418 (+ 6us) client_negotiation.cc:769] Sending connection context
0811 02:52:00.957473 (+ 55us) client_negotiation.cc:240] Negotiation successful
0811 02:52:00.957540 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":229,"thread_start_us":125,"threads_started":1}
I20260811 02:52:00.957696 15649 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:52:00.954551 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.138.194:44453 (local address 127.12.138.193:38359)
0811 02:52:00.954707 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0811 02:52:00.954711 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 02:52:00.954729 (+ 18us) server_negotiation.cc:407] Connection header received
0811 02:52:00.954776 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:52:00.954779 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:52:00.954820 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:52:00.954888 (+ 68us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 02:52:00.955145 (+ 257us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:52:00.955659 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:52:00.956542 (+ 883us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:52:00.956793 (+ 251us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:52:00.956907 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:52:00.957245 (+ 338us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 02:52:00.957325 (+ 80us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 02:52:00.957391 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0811 02:52:00.957508 (+ 117us) server_negotiation.cc:299] Negotiation successful
0811 02:52:00.957564 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260811 02:52:00.964301 15115 raft_consensus.cc:1261] T 55f73b1e296446a192974d93f0a7ef37 P c8aa4e1129d146d4a47f70f248771868 [term 1 FOLLOWER]: Refusing update from remote peer e6cbb5d02dcb40edb34ab5d41193496c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 02:52:00.964732 15438 raft_consensus.cc:1261] T 55f73b1e296446a192974d93f0a7ef37 P c0f254aed5f64d8b8f923c83b2b067b7 [term 1 FOLLOWER]: Refusing update from remote peer e6cbb5d02dcb40edb34ab5d41193496c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 02:52:00.964902 15643 consensus_queue.cc:1035] T 55f73b1e296446a192974d93f0a7ef37 P e6cbb5d02dcb40edb34ab5d41193496c [LEADER]: Connected to new peer: Peer: permanent_uuid: "c8aa4e1129d146d4a47f70f248771868" member_type: VOTER last_known_addr { host: "127.12.138.193" port: 38359 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 02:52:00.965250 15651 consensus_queue.cc:1035] T 55f73b1e296446a192974d93f0a7ef37 P e6cbb5d02dcb40edb34ab5d41193496c [LEADER]: Connected to new peer: Peer: permanent_uuid: "c0f254aed5f64d8b8f923c83b2b067b7" member_type: VOTER last_known_addr { host: "127.12.138.195" port: 42421 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 02:52:00.983225 15669 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:52:00.979534 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.138.194:33581 (local address 127.12.138.193:40975)
0811 02:52:00.979870 (+ 336us) negotiation.cc:107] Waiting for socket to connect
0811 02:52:00.979883 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 02:52:00.980009 (+ 126us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:52:00.980309 (+ 300us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:52:00.980313 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:52:00.980325 (+ 12us) client_negotiation.cc:189] Negotiated authn=TOKEN
0811 02:52:00.980435 (+ 110us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:52:00.980440 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:52:00.981094 (+ 654us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:52:00.981096 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:52:00.981862 (+ 766us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:52:00.981872 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:52:00.982450 (+ 578us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:52:00.982455 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:52:00.982548 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:52:00.982577 (+ 29us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0811 02:52:00.982930 (+ 353us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0811 02:52:00.982939 (+ 9us) client_negotiation.cc:769] Sending connection context
0811 02:52:00.983006 (+ 67us) client_negotiation.cc:240] Negotiation successful
0811 02:52:00.983090 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":268,"thread_start_us":106,"threads_started":1}
I20260811 02:52:00.983213 15656 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:52:00.979635 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.138.193:40975 (local address 127.12.138.194:33581)
0811 02:52:00.979782 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0811 02:52:00.979788 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 02:52:00.979965 (+ 177us) server_negotiation.cc:407] Connection header received
0811 02:52:00.980143 (+ 178us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:52:00.980147 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:52:00.980200 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:52:00.980281 (+ 81us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 02:52:00.980508 (+ 227us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:52:00.981003 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:52:00.982001 (+ 998us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:52:00.982312 (+ 311us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:52:00.982429 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:52:00.982714 (+ 285us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 02:52:00.982832 (+ 118us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 02:52:00.982920 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0811 02:52:00.983015 (+ 95us) server_negotiation.cc:299] Negotiation successful
0811 02:52:00.983067 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260811 02:52:01.008674 14933 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:50234:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 02:52:01.008972 14933 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:50234:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 02:52:01.011590 14933 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 60ef762b8fc246488dfb261f0b1e8420: Sending DeleteTablet for 3 replicas of tablet 55f73b1e296446a192974d93f0a7ef37
I20260811 02:52:01.012341 15090 tablet_service.cc:1514] Processing DeleteTablet for tablet 55f73b1e296446a192974d93f0a7ef37 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 02:52:01 UTC) from {username='kudu', principal='kudu/127.12.138.254@KRBTEST.COM'} at 127.0.0.1:37042
I20260811 02:52:01.012393 15421 tablet_service.cc:1514] Processing DeleteTablet for tablet 55f73b1e296446a192974d93f0a7ef37 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 02:52:01 UTC) from {username='kudu', principal='kudu/127.12.138.254@KRBTEST.COM'} at 127.0.0.1:40906
I20260811 02:52:01.012488 15258 tablet_service.cc:1514] Processing DeleteTablet for tablet 55f73b1e296446a192974d93f0a7ef37 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 02:52:01 UTC) from {username='kudu', principal='kudu/127.12.138.254@KRBTEST.COM'} at 127.0.0.1:53890
I20260811 02:52:01.012560 15624 tablet_replica.cc:330] T 55f73b1e296446a192974d93f0a7ef37 P c8aa4e1129d146d4a47f70f248771868: stopping tablet replica
I20260811 02:52:01.012621 15626 tablet_replica.cc:330] T 55f73b1e296446a192974d93f0a7ef37 P c0f254aed5f64d8b8f923c83b2b067b7: stopping tablet replica
I20260811 02:52:01.012696 15624 raft_consensus.cc:2229] T 55f73b1e296446a192974d93f0a7ef37 P c8aa4e1129d146d4a47f70f248771868 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 02:52:01.012737 15625 tablet_replica.cc:330] T 55f73b1e296446a192974d93f0a7ef37 P e6cbb5d02dcb40edb34ab5d41193496c: stopping tablet replica
I20260811 02:52:01.012737 15626 raft_consensus.cc:2229] T 55f73b1e296446a192974d93f0a7ef37 P c0f254aed5f64d8b8f923c83b2b067b7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 02:52:01.012818 15624 raft_consensus.cc:2258] T 55f73b1e296446a192974d93f0a7ef37 P c8aa4e1129d146d4a47f70f248771868 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 02:52:01.012831 15625 raft_consensus.cc:2229] T 55f73b1e296446a192974d93f0a7ef37 P e6cbb5d02dcb40edb34ab5d41193496c [term 1 LEADER]: Raft consensus shutting down.
I20260811 02:52:01.012854 15626 raft_consensus.cc:2258] T 55f73b1e296446a192974d93f0a7ef37 P c0f254aed5f64d8b8f923c83b2b067b7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 02:52:01.013211 15625 raft_consensus.cc:2258] T 55f73b1e296446a192974d93f0a7ef37 P e6cbb5d02dcb40edb34ab5d41193496c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 02:52:01.013922 15625 ts_tablet_manager.cc:1905] T 55f73b1e296446a192974d93f0a7ef37 P e6cbb5d02dcb40edb34ab5d41193496c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 02:52:01.013932 15626 ts_tablet_manager.cc:1905] T 55f73b1e296446a192974d93f0a7ef37 P c0f254aed5f64d8b8f923c83b2b067b7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 02:52:01.016120 15625 ts_tablet_manager.cc:1918] T 55f73b1e296446a192974d93f0a7ef37 P e6cbb5d02dcb40edb34ab5d41193496c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 02:52:01.016219 15625 log.cc:1196] T 55f73b1e296446a192974d93f0a7ef37 P e6cbb5d02dcb40edb34ab5d41193496c: Deleting WAL directory at /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786416707705494-12843-0/minicluster-data/ts-1/wal/wals/55f73b1e296446a192974d93f0a7ef37
I20260811 02:52:01.016485 15625 ts_tablet_manager.cc:1939] T 55f73b1e296446a192974d93f0a7ef37 P e6cbb5d02dcb40edb34ab5d41193496c: Deleting consensus metadata
I20260811 02:52:01.016935 15626 ts_tablet_manager.cc:1918] T 55f73b1e296446a192974d93f0a7ef37 P c0f254aed5f64d8b8f923c83b2b067b7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 02:52:01.017167 14918 catalog_manager.cc:4841] TS e6cbb5d02dcb40edb34ab5d41193496c (127.12.138.194:33581): tablet 55f73b1e296446a192974d93f0a7ef37 (table test-table [id=8cea614cd44d495b9377a1c9bdb29895]) successfully deleted
I20260811 02:52:01.017554 15624 ts_tablet_manager.cc:1905] T 55f73b1e296446a192974d93f0a7ef37 P c8aa4e1129d146d4a47f70f248771868: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 02:52:01.018553 14933 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:50234:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 02:52:01.019097 14933 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 02:52:01.020258 15624 ts_tablet_manager.cc:1918] T 55f73b1e296446a192974d93f0a7ef37 P c8aa4e1129d146d4a47f70f248771868: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 02:52:01.020356 15624 log.cc:1196] T 55f73b1e296446a192974d93f0a7ef37 P c8aa4e1129d146d4a47f70f248771868: Deleting WAL directory at /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786416707705494-12843-0/minicluster-data/ts-0/wal/wals/55f73b1e296446a192974d93f0a7ef37
I20260811 02:52:01.020874 15624 ts_tablet_manager.cc:1939] T 55f73b1e296446a192974d93f0a7ef37 P c8aa4e1129d146d4a47f70f248771868: Deleting consensus metadata
I20260811 02:52:01.021018 15626 log.cc:1196] T 55f73b1e296446a192974d93f0a7ef37 P c0f254aed5f64d8b8f923c83b2b067b7: Deleting WAL directory at /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786416707705494-12843-0/minicluster-data/ts-2/wal/wals/55f73b1e296446a192974d93f0a7ef37
I20260811 02:52:01.022758 14918 catalog_manager.cc:4841] TS c8aa4e1129d146d4a47f70f248771868 (127.12.138.193:38359): tablet 55f73b1e296446a192974d93f0a7ef37 (table test-table [id=8cea614cd44d495b9377a1c9bdb29895]) successfully deleted
I20260811 02:52:01.023298 15626 ts_tablet_manager.cc:1939] T 55f73b1e296446a192974d93f0a7ef37 P c0f254aed5f64d8b8f923c83b2b067b7: Deleting consensus metadata
I20260811 02:52:01.023787 15090 tablet_service.cc:1467] Processing CreateTablet for tablet 26da539e8cab431baafd3883cfd17516 (DEFAULT_TABLE table=test-table [id=d0cedbc9ecdd446e84ca8953dd26b9bc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 02:52:01.024062 15090 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 26da539e8cab431baafd3883cfd17516. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 02:52:01.024286 15421 tablet_service.cc:1467] Processing CreateTablet for tablet 26da539e8cab431baafd3883cfd17516 (DEFAULT_TABLE table=test-table [id=d0cedbc9ecdd446e84ca8953dd26b9bc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 02:52:01.024343 15258 tablet_service.cc:1467] Processing CreateTablet for tablet 26da539e8cab431baafd3883cfd17516 (DEFAULT_TABLE table=test-table [id=d0cedbc9ecdd446e84ca8953dd26b9bc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 02:52:01.024586 15421 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 26da539e8cab431baafd3883cfd17516. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 02:52:01.024607 15258 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 26da539e8cab431baafd3883cfd17516. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 02:52:01.026945 15638 tablet_bootstrap.cc:492] T 26da539e8cab431baafd3883cfd17516 P c8aa4e1129d146d4a47f70f248771868: Bootstrap starting.
I20260811 02:52:01.027249 15640 tablet_bootstrap.cc:492] T 26da539e8cab431baafd3883cfd17516 P c0f254aed5f64d8b8f923c83b2b067b7: Bootstrap starting.
I20260811 02:52:01.027814 15638 tablet_bootstrap.cc:654] T 26da539e8cab431baafd3883cfd17516 P c8aa4e1129d146d4a47f70f248771868: Neither blocks nor log segments found. Creating new log.
I20260811 02:52:01.028204 15640 tablet_bootstrap.cc:654] T 26da539e8cab431baafd3883cfd17516 P c0f254aed5f64d8b8f923c83b2b067b7: Neither blocks nor log segments found. Creating new log.
I20260811 02:52:01.028928 15637 tablet_bootstrap.cc:492] T 26da539e8cab431baafd3883cfd17516 P e6cbb5d02dcb40edb34ab5d41193496c: Bootstrap starting.
I20260811 02:52:01.029057 15638 tablet_bootstrap.cc:492] T 26da539e8cab431baafd3883cfd17516 P c8aa4e1129d146d4a47f70f248771868: No bootstrap required, opened a new log
I20260811 02:52:01.029132 15638 ts_tablet_manager.cc:1397] T 26da539e8cab431baafd3883cfd17516 P c8aa4e1129d146d4a47f70f248771868: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260811 02:52:01.029561 15638 raft_consensus.cc:348] T 26da539e8cab431baafd3883cfd17516 P c8aa4e1129d146d4a47f70f248771868 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8aa4e1129d146d4a47f70f248771868" member_type: VOTER last_known_addr { host: "127.12.138.193" port: 38359 } } peers { permanent_uuid: "e6cbb5d02dcb40edb34ab5d41193496c" member_type: VOTER last_known_addr { host: "127.12.138.194" port: 33581 } } peers { permanent_uuid: "c0f254aed5f64d8b8f923c83b2b067b7" member_type: VOTER last_known_addr { host: "127.12.138.195" port: 42421 } }
I20260811 02:52:01.029675 15638 raft_consensus.cc:374] T 26da539e8cab431baafd3883cfd17516 P c8aa4e1129d146d4a47f70f248771868 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 02:52:01.029704 15638 raft_consensus.cc:729] T 26da539e8cab431baafd3883cfd17516 P c8aa4e1129d146d4a47f70f248771868 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c8aa4e1129d146d4a47f70f248771868, State: Initialized, Role: FOLLOWER
I20260811 02:52:01.029829 15638 consensus_queue.cc:260] T 26da539e8cab431baafd3883cfd17516 P c8aa4e1129d146d4a47f70f248771868 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8aa4e1129d146d4a47f70f248771868" member_type: VOTER last_known_addr { host: "127.12.138.193" port: 38359 } } peers { permanent_uuid: "e6cbb5d02dcb40edb34ab5d41193496c" member_type: VOTER last_known_addr { host: "127.12.138.194" port: 33581 } } peers { permanent_uuid: "c0f254aed5f64d8b8f923c83b2b067b7" member_type: VOTER last_known_addr { host: "127.12.138.195" port: 42421 } }
I20260811 02:52:01.029899 15637 tablet_bootstrap.cc:654] T 26da539e8cab431baafd3883cfd17516 P e6cbb5d02dcb40edb34ab5d41193496c: Neither blocks nor log segments found. Creating new log.
I20260811 02:52:01.030079 15638 ts_tablet_manager.cc:1428] T 26da539e8cab431baafd3883cfd17516 P c8aa4e1129d146d4a47f70f248771868: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260811 02:52:01.030783 14919 catalog_manager.cc:4841] TS c0f254aed5f64d8b8f923c83b2b067b7 (127.12.138.195:42421): tablet 55f73b1e296446a192974d93f0a7ef37 (table test-table [id=8cea614cd44d495b9377a1c9bdb29895]) successfully deleted
I20260811 02:52:01.031792 15640 tablet_bootstrap.cc:492] T 26da539e8cab431baafd3883cfd17516 P c0f254aed5f64d8b8f923c83b2b067b7: No bootstrap required, opened a new log
I20260811 02:52:01.031883 15640 ts_tablet_manager.cc:1397] T 26da539e8cab431baafd3883cfd17516 P c0f254aed5f64d8b8f923c83b2b067b7: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20260811 02:52:01.031908 15637 tablet_bootstrap.cc:492] T 26da539e8cab431baafd3883cfd17516 P e6cbb5d02dcb40edb34ab5d41193496c: No bootstrap required, opened a new log
I20260811 02:52:01.032019 15637 ts_tablet_manager.cc:1397] T 26da539e8cab431baafd3883cfd17516 P e6cbb5d02dcb40edb34ab5d41193496c: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260811 02:52:01.032270 15640 raft_consensus.cc:348] T 26da539e8cab431baafd3883cfd17516 P c0f254aed5f64d8b8f923c83b2b067b7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8aa4e1129d146d4a47f70f248771868" member_type: VOTER last_known_addr { host: "127.12.138.193" port: 38359 } } peers { permanent_uuid: "e6cbb5d02dcb40edb34ab5d41193496c" member_type: VOTER last_known_addr { host: "127.12.138.194" port: 33581 } } peers { permanent_uuid: "c0f254aed5f64d8b8f923c83b2b067b7" member_type: VOTER last_known_addr { host: "127.12.138.195" port: 42421 } }
I20260811 02:52:01.032398 15640 raft_consensus.cc:374] T 26da539e8cab431baafd3883cfd17516 P c0f254aed5f64d8b8f923c83b2b067b7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 02:52:01.032433 15640 raft_consensus.cc:729] T 26da539e8cab431baafd3883cfd17516 P c0f254aed5f64d8b8f923c83b2b067b7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c0f254aed5f64d8b8f923c83b2b067b7, State: Initialized, Role: FOLLOWER
I20260811 02:52:01.032550 15640 consensus_queue.cc:260] T 26da539e8cab431baafd3883cfd17516 P c0f254aed5f64d8b8f923c83b2b067b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8aa4e1129d146d4a47f70f248771868" member_type: VOTER last_known_addr { host: "127.12.138.193" port: 38359 } } peers { permanent_uuid: "e6cbb5d02dcb40edb34ab5d41193496c" member_type: VOTER last_known_addr { host: "127.12.138.194" port: 33581 } } peers { permanent_uuid: "c0f254aed5f64d8b8f923c83b2b067b7" member_type: VOTER last_known_addr { host: "127.12.138.195" port: 42421 } }
I20260811 02:52:01.032544 15637 raft_consensus.cc:348] T 26da539e8cab431baafd3883cfd17516 P e6cbb5d02dcb40edb34ab5d41193496c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8aa4e1129d146d4a47f70f248771868" member_type: VOTER last_known_addr { host: "127.12.138.193" port: 38359 } } peers { permanent_uuid: "e6cbb5d02dcb40edb34ab5d41193496c" member_type: VOTER last_known_addr { host: "127.12.138.194" port: 33581 } } peers { permanent_uuid: "c0f254aed5f64d8b8f923c83b2b067b7" member_type: VOTER last_known_addr { host: "127.12.138.195" port: 42421 } }
I20260811 02:52:01.032697 15637 raft_consensus.cc:374] T 26da539e8cab431baafd3883cfd17516 P e6cbb5d02dcb40edb34ab5d41193496c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 02:52:01.032740 15637 raft_consensus.cc:729] T 26da539e8cab431baafd3883cfd17516 P e6cbb5d02dcb40edb34ab5d41193496c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e6cbb5d02dcb40edb34ab5d41193496c, State: Initialized, Role: FOLLOWER
I20260811 02:52:01.032801 15640 ts_tablet_manager.cc:1428] T 26da539e8cab431baafd3883cfd17516 P c0f254aed5f64d8b8f923c83b2b067b7: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260811 02:52:01.032887 15637 consensus_queue.cc:260] T 26da539e8cab431baafd3883cfd17516 P e6cbb5d02dcb40edb34ab5d41193496c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8aa4e1129d146d4a47f70f248771868" member_type: VOTER last_known_addr { host: "127.12.138.193" port: 38359 } } peers { permanent_uuid: "e6cbb5d02dcb40edb34ab5d41193496c" member_type: VOTER last_known_addr { host: "127.12.138.194" port: 33581 } } peers { permanent_uuid: "c0f254aed5f64d8b8f923c83b2b067b7" member_type: VOTER last_known_addr { host: "127.12.138.195" port: 42421 } }
I20260811 02:52:01.033231 15637 ts_tablet_manager.cc:1428] T 26da539e8cab431baafd3883cfd17516 P e6cbb5d02dcb40edb34ab5d41193496c: Time spent starting tablet: real 0.001s user 0.002s sys 0.000s
I20260811 02:52:01.062475 15608 raft_consensus.cc:482] T 26da539e8cab431baafd3883cfd17516 P c8aa4e1129d146d4a47f70f248771868 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 02:52:01.062625 15608 raft_consensus.cc:504] T 26da539e8cab431baafd3883cfd17516 P c8aa4e1129d146d4a47f70f248771868 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8aa4e1129d146d4a47f70f248771868" member_type: VOTER last_known_addr { host: "127.12.138.193" port: 38359 } } peers { permanent_uuid: "e6cbb5d02dcb40edb34ab5d41193496c" member_type: VOTER last_known_addr { host: "127.12.138.194" port: 33581 } } peers { permanent_uuid: "c0f254aed5f64d8b8f923c83b2b067b7" member_type: VOTER last_known_addr { host: "127.12.138.195" port: 42421 } }
I20260811 02:52:01.063235 15608 leader_election.cc:288] T 26da539e8cab431baafd3883cfd17516 P c8aa4e1129d146d4a47f70f248771868 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e6cbb5d02dcb40edb34ab5d41193496c (127.12.138.194:33581), c0f254aed5f64d8b8f923c83b2b067b7 (127.12.138.195:42421)
I20260811 02:52:01.063719 15280 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "26da539e8cab431baafd3883cfd17516" candidate_uuid: "c8aa4e1129d146d4a47f70f248771868" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e6cbb5d02dcb40edb34ab5d41193496c" is_pre_election: true
I20260811 02:52:01.063766 15438 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "26da539e8cab431baafd3883cfd17516" candidate_uuid: "c8aa4e1129d146d4a47f70f248771868" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c0f254aed5f64d8b8f923c83b2b067b7" is_pre_election: true
I20260811 02:52:01.063921 15280 raft_consensus.cc:2454] T 26da539e8cab431baafd3883cfd17516 P e6cbb5d02dcb40edb34ab5d41193496c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c8aa4e1129d146d4a47f70f248771868 in term 0.
I20260811 02:52:01.063987 15438 raft_consensus.cc:2454] T 26da539e8cab431baafd3883cfd17516 P c0f254aed5f64d8b8f923c83b2b067b7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c8aa4e1129d146d4a47f70f248771868 in term 0.
I20260811 02:52:01.064383 15036 leader_election.cc:302] T 26da539e8cab431baafd3883cfd17516 P c8aa4e1129d146d4a47f70f248771868 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c8aa4e1129d146d4a47f70f248771868, e6cbb5d02dcb40edb34ab5d41193496c; no voters:
I20260811 02:52:01.064581 15608 raft_consensus.cc:2781] T 26da539e8cab431baafd3883cfd17516 P c8aa4e1129d146d4a47f70f248771868 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 02:52:01.064642 15608 raft_consensus.cc:482] T 26da539e8cab431baafd3883cfd17516 P c8aa4e1129d146d4a47f70f248771868 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 02:52:01.064699 15608 raft_consensus.cc:3037] T 26da539e8cab431baafd3883cfd17516 P c8aa4e1129d146d4a47f70f248771868 [term 0 FOLLOWER]: Advancing to term 1
I20260811 02:52:01.066455 15608 raft_consensus.cc:504] T 26da539e8cab431baafd3883cfd17516 P c8aa4e1129d146d4a47f70f248771868 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8aa4e1129d146d4a47f70f248771868" member_type: VOTER last_known_addr { host: "127.12.138.193" port: 38359 } } peers { permanent_uuid: "e6cbb5d02dcb40edb34ab5d41193496c" member_type: VOTER last_known_addr { host: "127.12.138.194" port: 33581 } } peers { permanent_uuid: "c0f254aed5f64d8b8f923c83b2b067b7" member_type: VOTER last_known_addr { host: "127.12.138.195" port: 42421 } }
I20260811 02:52:01.066891 15608 leader_election.cc:288] T 26da539e8cab431baafd3883cfd17516 P c8aa4e1129d146d4a47f70f248771868 [CANDIDATE]: Term 1 election: Requested vote from peers e6cbb5d02dcb40edb34ab5d41193496c (127.12.138.194:33581), c0f254aed5f64d8b8f923c83b2b067b7 (127.12.138.195:42421)
I20260811 02:52:01.067267 15280 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "26da539e8cab431baafd3883cfd17516" candidate_uuid: "c8aa4e1129d146d4a47f70f248771868" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e6cbb5d02dcb40edb34ab5d41193496c"
I20260811 02:52:01.067294 15438 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "26da539e8cab431baafd3883cfd17516" candidate_uuid: "c8aa4e1129d146d4a47f70f248771868" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c0f254aed5f64d8b8f923c83b2b067b7"
I20260811 02:52:01.067422 15438 raft_consensus.cc:3037] T 26da539e8cab431baafd3883cfd17516 P c0f254aed5f64d8b8f923c83b2b067b7 [term 0 FOLLOWER]: Advancing to term 1
I20260811 02:52:01.067420 15280 raft_consensus.cc:3037] T 26da539e8cab431baafd3883cfd17516 P e6cbb5d02dcb40edb34ab5d41193496c [term 0 FOLLOWER]: Advancing to term 1
I20260811 02:52:01.068722 15438 raft_consensus.cc:2454] T 26da539e8cab431baafd3883cfd17516 P c0f254aed5f64d8b8f923c83b2b067b7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c8aa4e1129d146d4a47f70f248771868 in term 1.
I20260811 02:52:01.068722 15280 raft_consensus.cc:2454] T 26da539e8cab431baafd3883cfd17516 P e6cbb5d02dcb40edb34ab5d41193496c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c8aa4e1129d146d4a47f70f248771868 in term 1.
I20260811 02:52:01.069056 15038 leader_election.cc:302] T 26da539e8cab431baafd3883cfd17516 P c8aa4e1129d146d4a47f70f248771868 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c0f254aed5f64d8b8f923c83b2b067b7, c8aa4e1129d146d4a47f70f248771868; no voters:
I20260811 02:52:01.069357 15608 raft_consensus.cc:2781] T 26da539e8cab431baafd3883cfd17516 P c8aa4e1129d146d4a47f70f248771868 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 02:52:01.069489 15608 raft_consensus.cc:686] T 26da539e8cab431baafd3883cfd17516 P c8aa4e1129d146d4a47f70f248771868 [term 1 LEADER]: Becoming Leader. State: Replica: c8aa4e1129d146d4a47f70f248771868, State: Running, Role: LEADER
I20260811 02:52:01.069649 15608 consensus_queue.cc:237] T 26da539e8cab431baafd3883cfd17516 P c8aa4e1129d146d4a47f70f248771868 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8aa4e1129d146d4a47f70f248771868" member_type: VOTER last_known_addr { host: "127.12.138.193" port: 38359 } } peers { permanent_uuid: "e6cbb5d02dcb40edb34ab5d41193496c" member_type: VOTER last_known_addr { host: "127.12.138.194" port: 33581 } } peers { permanent_uuid: "c0f254aed5f64d8b8f923c83b2b067b7" member_type: VOTER last_known_addr { host: "127.12.138.195" port: 42421 } }
I20260811 02:52:01.071637 14928 catalog_manager.cc:5495] T 26da539e8cab431baafd3883cfd17516 P c8aa4e1129d146d4a47f70f248771868 reported cstate change: term changed from 0 to 1, leader changed from <none> to c8aa4e1129d146d4a47f70f248771868 (127.12.138.193). New cstate: current_term: 1 leader_uuid: "c8aa4e1129d146d4a47f70f248771868" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8aa4e1129d146d4a47f70f248771868" member_type: VOTER last_known_addr { host: "127.12.138.193" port: 38359 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e6cbb5d02dcb40edb34ab5d41193496c" member_type: VOTER last_known_addr { host: "127.12.138.194" port: 33581 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c0f254aed5f64d8b8f923c83b2b067b7" member_type: VOTER last_known_addr { host: "127.12.138.195" port: 42421 } health_report { overall_health: UNKNOWN } } }
I20260811 02:52:01.088909 15649 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:52:01.085034 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37066 (local address 127.12.138.193:38359)
0811 02:52:01.085188 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0811 02:52:01.085193 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 02:52:01.085344 (+ 151us) server_negotiation.cc:407] Connection header received
0811 02:52:01.085438 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:52:01.085441 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:52:01.085495 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:52:01.085576 (+ 81us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 02:52:01.085900 (+ 324us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:52:01.086724 (+ 824us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:52:01.087654 (+ 930us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:52:01.088001 (+ 347us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:52:01.088109 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:52:01.088376 (+ 267us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 02:52:01.088462 (+ 86us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 02:52:01.088539 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0811 02:52:01.088708 (+ 169us) server_negotiation.cc:299] Negotiation successful
0811 02:52:01.088765 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260811 02:52:01.095721 15438 raft_consensus.cc:1261] T 26da539e8cab431baafd3883cfd17516 P c0f254aed5f64d8b8f923c83b2b067b7 [term 1 FOLLOWER]: Refusing update from remote peer c8aa4e1129d146d4a47f70f248771868: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 02:52:01.095892 15280 raft_consensus.cc:1261] T 26da539e8cab431baafd3883cfd17516 P e6cbb5d02dcb40edb34ab5d41193496c [term 1 FOLLOWER]: Refusing update from remote peer c8aa4e1129d146d4a47f70f248771868: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 02:52:01.096170 15608 consensus_queue.cc:1035] T 26da539e8cab431baafd3883cfd17516 P c8aa4e1129d146d4a47f70f248771868 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c0f254aed5f64d8b8f923c83b2b067b7" member_type: VOTER last_known_addr { host: "127.12.138.195" port: 42421 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 02:52:01.096352 15600 consensus_queue.cc:1035] T 26da539e8cab431baafd3883cfd17516 P c8aa4e1129d146d4a47f70f248771868 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e6cbb5d02dcb40edb34ab5d41193496c" member_type: VOTER last_known_addr { host: "127.12.138.194" port: 33581 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260811 02:52:01.112377 15601 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260811 02:52:02.675673 14928 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:50234:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 02:52:02.675892 14928 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:50234:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 02:52:02.678891 14928 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 60ef762b8fc246488dfb261f0b1e8420: Sending DeleteTablet for 3 replicas of tablet 26da539e8cab431baafd3883cfd17516
I20260811 02:52:02.679527 15090 tablet_service.cc:1514] Processing DeleteTablet for tablet 26da539e8cab431baafd3883cfd17516 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 02:52:02 UTC) from {username='kudu', principal='kudu/127.12.138.254@KRBTEST.COM'} at 127.0.0.1:37042
I20260811 02:52:02.679549 15258 tablet_service.cc:1514] Processing DeleteTablet for tablet 26da539e8cab431baafd3883cfd17516 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 02:52:02 UTC) from {username='kudu', principal='kudu/127.12.138.254@KRBTEST.COM'} at 127.0.0.1:53890
I20260811 02:52:02.679922 15421 tablet_service.cc:1514] Processing DeleteTablet for tablet 26da539e8cab431baafd3883cfd17516 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 02:52:02 UTC) from {username='kudu', principal='kudu/127.12.138.254@KRBTEST.COM'} at 127.0.0.1:40906
I20260811 02:52:02.680001 12843 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDawEwS/build/debug/bin/kudu with pid 15015
I20260811 02:52:02.685868 15699 tablet_replica.cc:330] T 26da539e8cab431baafd3883cfd17516 P e6cbb5d02dcb40edb34ab5d41193496c: stopping tablet replica
I20260811 02:52:02.686112 15699 raft_consensus.cc:2229] T 26da539e8cab431baafd3883cfd17516 P e6cbb5d02dcb40edb34ab5d41193496c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 02:52:02.686288 15699 raft_consensus.cc:2258] T 26da539e8cab431baafd3883cfd17516 P e6cbb5d02dcb40edb34ab5d41193496c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 02:52:02.686600 15701 tablet_replica.cc:330] T 26da539e8cab431baafd3883cfd17516 P c0f254aed5f64d8b8f923c83b2b067b7: stopping tablet replica
I20260811 02:52:02.686766 15701 raft_consensus.cc:2229] T 26da539e8cab431baafd3883cfd17516 P c0f254aed5f64d8b8f923c83b2b067b7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 02:52:02.686904 15701 raft_consensus.cc:2258] T 26da539e8cab431baafd3883cfd17516 P c0f254aed5f64d8b8f923c83b2b067b7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 02:52:02.687624 15701 ts_tablet_manager.cc:1905] T 26da539e8cab431baafd3883cfd17516 P c0f254aed5f64d8b8f923c83b2b067b7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 02:52:02.687690 15699 ts_tablet_manager.cc:1905] T 26da539e8cab431baafd3883cfd17516 P e6cbb5d02dcb40edb34ab5d41193496c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 02:52:02.690045 15701 ts_tablet_manager.cc:1918] T 26da539e8cab431baafd3883cfd17516 P c0f254aed5f64d8b8f923c83b2b067b7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260811 02:52:02.690156 15701 log.cc:1196] T 26da539e8cab431baafd3883cfd17516 P c0f254aed5f64d8b8f923c83b2b067b7: Deleting WAL directory at /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786416707705494-12843-0/minicluster-data/ts-2/wal/wals/26da539e8cab431baafd3883cfd17516
I20260811 02:52:02.690421 15701 ts_tablet_manager.cc:1939] T 26da539e8cab431baafd3883cfd17516 P c0f254aed5f64d8b8f923c83b2b067b7: Deleting consensus metadata
I20260811 02:52:02.691182 14919 catalog_manager.cc:4841] TS c0f254aed5f64d8b8f923c83b2b067b7 (127.12.138.195:42421): tablet 26da539e8cab431baafd3883cfd17516 (table test-table [id=d0cedbc9ecdd446e84ca8953dd26b9bc]) successfully deleted
I20260811 02:52:02.691480 15699 ts_tablet_manager.cc:1918] T 26da539e8cab431baafd3883cfd17516 P e6cbb5d02dcb40edb34ab5d41193496c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260811 02:52:02.691581 15699 log.cc:1196] T 26da539e8cab431baafd3883cfd17516 P e6cbb5d02dcb40edb34ab5d41193496c: Deleting WAL directory at /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786416707705494-12843-0/minicluster-data/ts-1/wal/wals/26da539e8cab431baafd3883cfd17516
I20260811 02:52:02.691905 15699 ts_tablet_manager.cc:1939] T 26da539e8cab431baafd3883cfd17516 P e6cbb5d02dcb40edb34ab5d41193496c: Deleting consensus metadata
I20260811 02:52:02.692821 14918 catalog_manager.cc:4841] TS e6cbb5d02dcb40edb34ab5d41193496c (127.12.138.194:33581): tablet 26da539e8cab431baafd3883cfd17516 (table test-table [id=d0cedbc9ecdd446e84ca8953dd26b9bc]) successfully deleted
W20260811 02:52:02.696336 14918 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.12.138.193:38359 (error 108)
I20260811 02:52:02.696548 12843 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDawEwS/build/debug/bin/kudu with pid 15179
I20260811 02:52:02.700802 15703 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:52:02.700301 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.138.193:38359 (local address 127.0.0.1:58476)
0811 02:52:02.700588 (+ 287us) negotiation.cc:107] Waiting for socket to connect
0811 02:52:02.700687 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.138.193:38359: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":197,"thread_start_us":122,"threads_started":1}
W20260811 02:52:02.700976 14918 catalog_manager.cc:4584] TS c8aa4e1129d146d4a47f70f248771868 (127.12.138.193:38359): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 26da539e8cab431baafd3883cfd17516: Network error: Client connection negotiation failed: client connection to 127.12.138.193:38359: connect: Connection refused (error 111)
I20260811 02:52:02.708195 12843 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDawEwS/build/debug/bin/kudu with pid 15341
I20260811 02:52:02.718636 12843 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDawEwS/build/debug/bin/kudu with pid 14896
2026-08-11T02:52:02Z chronyd exiting
[ OK ] SecurityITest.TxnSmokeWithDifferentUserTypes (6670 ms)
[ RUN ] SecurityITest.TestNoKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786416707705494-12843-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 02:52:02 dist-test-slave-9gcw krb5kdc[15707](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 02:52:02 dist-test-slave-9gcw krb5kdc[15707](info): set up 2 sockets
Aug 11 02:52:02 dist-test-slave-9gcw krb5kdc[15707](info): commencing operation
krb5kdc: starting...
W20260811 02:52:03.944561 12843 mini_kdc.cc:121] Time spent starting KDC: real 1.198s user 0.007s sys 0.001s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786416707705494-12843-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786416707705494-12843-0/krb5kdc/test-user.keytab.
Aug 11 02:52:04 dist-test-slave-9gcw krb5kdc[15707](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416724, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T02:52:04Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T02:52:04Z Disabled control of system clock
WARNING: no policy specified for kudu/127.12.138.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.138.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.138.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786416707705494-12843-0/krb5kdc/kudu_127.12.138.254.keytab.
Entry for principal kudu/127.12.138.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786416707705494-12843-0/krb5kdc/kudu_127.12.138.254.keytab.
I20260811 02:52:04.093708 12843 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDawEwS/build/debug/bin/kudu
/tmp/dist-test-taskDawEwS/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786416707705494-12843-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786416707705494-12843-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786416707705494-12843-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786416707705494-12843-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.138.254:33819
--webserver_interface=127.12.138.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786416707705494-12843-0/krb5kdc/kudu_127.12.138.254.keytab
--principal=kudu/127.12.138.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.138.212:33935
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.12.138.254:33819
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786416707705494-12843-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786416707705494-12843-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786416707705494-12843-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 02:52:04.215384 15723 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 02:52:04.215667 15723 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 02:52:04.215709 15723 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 02:52:04.219830 15723 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 02:52:04.219894 15723 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 02:52:04.219914 15723 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 02:52:04.219930 15723 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 02:52:04.219946 15723 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 02:52:04.225396 15723 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.138.212:33935
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786416707705494-12843-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786416707705494-12843-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.12.138.254:33819
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786416707705494-12843-0/krb5kdc/kudu_127.12.138.254.keytab
--principal=kudu/127.12.138.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.138.254:33819
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786416707705494-12843-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.12.138.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15723
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786416707705494-12843-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 02:09:49 UTC on ccycloud.quasar-fsnqqc.root.comops.site
build id 4579570
I20260811 02:52:04.226493 15723 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 02:52:04.227567 15723 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 02:52:04.236984 15732 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:52:04.237087 15729 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:52:04.237046 15723 server_base.cc:1034] running on GCE node
W20260811 02:52:04.237243 15730 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:52:04.237975 15723 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 02:52:04.239202 15723 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 02:52:04.240483 15723 hybrid_clock.cc:648] HybridClock initialized: now 1786416724240467 us; error 36 us; skew 500 ppm
Aug 11 02:52:04 dist-test-slave-9gcw krb5kdc[15707](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416724, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.138.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 02:52:04.243660 15723 init.cc:377] Logged in from keytab as kudu/127.12.138.254@KRBTEST.COM (short username kudu)
I20260811 02:52:04.244648 15723 webserver.cc:460] Webserver started at http://127.12.138.254:45833/ using document root <none> and password file <none>
I20260811 02:52:04.245306 15723 fs_manager.cc:359] Metadata directory not provided
I20260811 02:52:04.245407 15723 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 02:52:04.245663 15723 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 02:52:04.247782 15723 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786416707705494-12843-0/minicluster-data/master-0/data/instance:
uuid: "36f6a5bca4434945b588106049883a65"
format_stamp: "Formatted at 2026-08-11 02:52:04 on dist-test-slave-9gcw"
server_key: "528df3dfc685bdba8e54f5fc4d9287da"
server_key_iv: "12ec7f760d60191a6c57aa5d4f0cb938"
server_key_version: "encryptionkey@0"
I20260811 02:52:04.248389 15723 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786416707705494-12843-0/minicluster-data/master-0/wal/instance:
uuid: "36f6a5bca4434945b588106049883a65"
format_stamp: "Formatted at 2026-08-11 02:52:04 on dist-test-slave-9gcw"
server_key: "528df3dfc685bdba8e54f5fc4d9287da"
server_key_iv: "12ec7f760d60191a6c57aa5d4f0cb938"
server_key_version: "encryptionkey@0"
I20260811 02:52:04.252324 15723 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.001s
I20260811 02:52:04.254933 15741 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 02:52:04.256315 15723 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260811 02:52:04.256461 15723 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786416707705494-12843-0/minicluster-data/master-0/data,/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786416707705494-12843-0/minicluster-data/master-0/wal
uuid: "36f6a5bca4434945b588106049883a65"
format_stamp: "Formatted at 2026-08-11 02:52:04 on dist-test-slave-9gcw"
server_key: "528df3dfc685bdba8e54f5fc4d9287da"
server_key_iv: "12ec7f760d60191a6c57aa5d4f0cb938"
server_key_version: "encryptionkey@0"
I20260811 02:52:04.256579 15723 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786416707705494-12843-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786416707705494-12843-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786416707705494-12843-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 02:52:04.290418 15723 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 02:52:04.294152 15723 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 02:52:04.294391 15723 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 02:52:04.302654 15723 rpc_server.cc:307] RPC server started. Bound to: 127.12.138.254:33819
I20260811 02:52:04.302681 15808 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.138.254:33819 every 8 connection(s)
I20260811 02:52:04.303669 15723 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786416707705494-12843-0/minicluster-data/master-0/data/info.pb
I20260811 02:52:04.306795 15809 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 02:52:04.312446 12843 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDawEwS/build/debug/bin/kudu as pid 15723
I20260811 02:52:04.312613 12843 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786416707705494-12843-0/minicluster-data/master-0/wal/instance
I20260811 02:52:04.312999 12843 external_mini_cluster.cc:1428] Setting key 78a7d9f5ecaf9790a47edfd667b8adf0
I20260811 02:52:04.313858 15809 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 36f6a5bca4434945b588106049883a65: Bootstrap starting.
I20260811 02:52:04.316726 15809 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 36f6a5bca4434945b588106049883a65: Neither blocks nor log segments found. Creating new log.
I20260811 02:52:04.317690 15809 log.cc:824] T 00000000000000000000000000000000 P 36f6a5bca4434945b588106049883a65: Log is configured to *not* fsync() on all Append() calls
I20260811 02:52:04.319963 15809 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 36f6a5bca4434945b588106049883a65: No bootstrap required, opened a new log
Aug 11 02:52:04 dist-test-slave-9gcw krb5kdc[15707](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416724, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.12.138.254@KRBTEST.COM
I20260811 02:52:04.324834 15809 raft_consensus.cc:348] T 00000000000000000000000000000000 P 36f6a5bca4434945b588106049883a65 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "36f6a5bca4434945b588106049883a65" member_type: VOTER last_known_addr { host: "127.12.138.254" port: 33819 } }
I20260811 02:52:04.325263 15809 raft_consensus.cc:374] T 00000000000000000000000000000000 P 36f6a5bca4434945b588106049883a65 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 02:52:04.325347 15809 raft_consensus.cc:729] T 00000000000000000000000000000000 P 36f6a5bca4434945b588106049883a65 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 36f6a5bca4434945b588106049883a65, State: Initialized, Role: FOLLOWER
I20260811 02:52:04.325896 15809 consensus_queue.cc:260] T 00000000000000000000000000000000 P 36f6a5bca4434945b588106049883a65 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "36f6a5bca4434945b588106049883a65" member_type: VOTER last_known_addr { host: "127.12.138.254" port: 33819 } }
I20260811 02:52:04.326045 15809 raft_consensus.cc:388] T 00000000000000000000000000000000 P 36f6a5bca4434945b588106049883a65 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 02:52:04.326200 15809 raft_consensus.cc:482] T 00000000000000000000000000000000 P 36f6a5bca4434945b588106049883a65 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 02:52:04.326320 15809 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 36f6a5bca4434945b588106049883a65 [term 0 FOLLOWER]: Advancing to term 1
I20260811 02:52:04.327456 15809 raft_consensus.cc:504] T 00000000000000000000000000000000 P 36f6a5bca4434945b588106049883a65 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "36f6a5bca4434945b588106049883a65" member_type: VOTER last_known_addr { host: "127.12.138.254" port: 33819 } }
I20260811 02:52:04.327858 15809 leader_election.cc:302] T 00000000000000000000000000000000 P 36f6a5bca4434945b588106049883a65 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 36f6a5bca4434945b588106049883a65; no voters:
I20260811 02:52:04.328300 15809 leader_election.cc:288] T 00000000000000000000000000000000 P 36f6a5bca4434945b588106049883a65 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 02:52:04.328572 15814 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 36f6a5bca4434945b588106049883a65 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 02:52:04.328882 15814 raft_consensus.cc:686] T 00000000000000000000000000000000 P 36f6a5bca4434945b588106049883a65 [term 1 LEADER]: Becoming Leader. State: Replica: 36f6a5bca4434945b588106049883a65, State: Running, Role: LEADER
I20260811 02:52:04.329396 15814 consensus_queue.cc:237] T 00000000000000000000000000000000 P 36f6a5bca4434945b588106049883a65 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "36f6a5bca4434945b588106049883a65" member_type: VOTER last_known_addr { host: "127.12.138.254" port: 33819 } }
I20260811 02:52:04.329535 15812 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:52:04.314808 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42774 (local address 127.12.138.254:33819)
0811 02:52:04.315313 (+ 505us) server_negotiation.cc:206] Beginning negotiation
0811 02:52:04.315328 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0811 02:52:04.315379 (+ 51us) server_negotiation.cc:407] Connection header received
0811 02:52:04.316105 (+ 726us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:52:04.316125 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:52:04.316464 (+ 339us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:52:04.316832 (+ 368us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:52:04.317710 (+ 878us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:52:04.318965 (+ 1255us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:52:04.319837 (+ 872us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:52:04.320374 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:52:04.320548 (+ 174us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:52:04.323842 (+ 3294us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:52:04.323848 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:52:04.323862 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:52:04.323890 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:52:04.326047 (+ 2157us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:52:04.326636 (+ 589us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:52:04.326642 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:52:04.326648 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:52:04.326779 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:52:04.327089 (+ 310us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:52:04.327093 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:52:04.327095 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:52:04.327517 (+ 422us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:52:04.327809 (+ 292us) server_negotiation.cc:1035] Waiting for connection context
0811 02:52:04.328125 (+ 316us) server_negotiation.cc:299] Negotiation successful
0811 02:52:04.328512 (+ 387us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":356,"thread_start_us":157,"threads_started":1}
I20260811 02:52:04.329895 15809 sys_catalog.cc:564] T 00000000000000000000000000000000 P 36f6a5bca4434945b588106049883a65 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 02:52:04.330797 15814 sys_catalog.cc:455] T 00000000000000000000000000000000 P 36f6a5bca4434945b588106049883a65 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 36f6a5bca4434945b588106049883a65. Latest consensus state: current_term: 1 leader_uuid: "36f6a5bca4434945b588106049883a65" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "36f6a5bca4434945b588106049883a65" member_type: VOTER last_known_addr { host: "127.12.138.254" port: 33819 } } }
I20260811 02:52:04.330958 15814 sys_catalog.cc:458] T 00000000000000000000000000000000 P 36f6a5bca4434945b588106049883a65 [sys.catalog]: This master's current role is: LEADER
I20260811 02:52:04.331223 15815 sys_catalog.cc:455] T 00000000000000000000000000000000 P 36f6a5bca4434945b588106049883a65 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "36f6a5bca4434945b588106049883a65" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "36f6a5bca4434945b588106049883a65" member_type: VOTER last_known_addr { host: "127.12.138.254" port: 33819 } } }
I20260811 02:52:04.331315 15815 sys_catalog.cc:458] T 00000000000000000000000000000000 P 36f6a5bca4434945b588106049883a65 [sys.catalog]: This master's current role is: LEADER
I20260811 02:52:04.332059 15821 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 02:52:04.335374 15821 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 02:52:04.341290 15821 catalog_manager.cc:1347] Generated new cluster ID: db9e3da1b6a94ac594f9a37b840fbe09
I20260811 02:52:04.341380 15821 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 02:52:04.356334 15821 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 02:52:04.357409 15821 catalog_manager.cc:1504] Loading token signing keys...
I20260811 02:52:04.369547 15821 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 36f6a5bca4434945b588106049883a65: Generated new TSK 0
I20260811 02:52:04.370323 15821 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.12.138.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.138.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.138.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786416707705494-12843-0/krb5kdc/kudu_127.12.138.193.keytab.
Entry for principal kudu/127.12.138.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786416707705494-12843-0/krb5kdc/kudu_127.12.138.193.keytab.
I20260811 02:52:04.412580 12843 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDawEwS/build/debug/bin/kudu
/tmp/dist-test-taskDawEwS/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786416707705494-12843-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786416707705494-12843-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786416707705494-12843-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786416707705494-12843-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.138.193:0
--local_ip_for_outbound_sockets=127.12.138.193
--webserver_interface=127.12.138.193
--webserver_port=0
--tserver_master_addrs=127.12.138.254:33819
--keytab_file=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786416707705494-12843-0/krb5kdc/kudu_127.12.138.193.keytab
--principal=kudu/127.12.138.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.138.212:33935
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786416707705494-12843-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786416707705494-12843-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786416707705494-12843-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 02:52:04.551395 15835 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 02:52:04.551729 15835 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 02:52:04.551811 15835 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 02:52:04.557003 15835 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 02:52:04.557124 15835 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 02:52:04.557255 15835 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.138.193
I20260811 02:52:04.564325 15835 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.138.212:33935
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786416707705494-12843-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786416707705494-12843-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786416707705494-12843-0/krb5kdc/kudu_127.12.138.193.keytab
--principal=kudu/127.12.138.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.138.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786416707705494-12843-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.12.138.193
--webserver_port=0
--tserver_master_addrs=127.12.138.254:33819
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15835
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.138.193
--log_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786416707705494-12843-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 02:09:49 UTC on ccycloud.quasar-fsnqqc.root.comops.site
build id 4579570
I20260811 02:52:04.565659 15835 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 02:52:04.566926 15835 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 02:52:04.577834 15841 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:52:04.577841 15843 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:52:04.577996 15835 server_base.cc:1034] running on GCE node
W20260811 02:52:04.577845 15840 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:52:04.578634 15835 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 02:52:04.579433 15835 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 02:52:04.580667 15835 hybrid_clock.cc:648] HybridClock initialized: now 1786416724580641 us; error 54 us; skew 500 ppm
Aug 11 02:52:04 dist-test-slave-9gcw krb5kdc[15707](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416724, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.138.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 02:52:04.583747 15835 init.cc:377] Logged in from keytab as kudu/127.12.138.193@KRBTEST.COM (short username kudu)
I20260811 02:52:04.584704 15835 webserver.cc:460] Webserver started at http://127.12.138.193:45219/ using document root <none> and password file <none>
I20260811 02:52:04.585378 15835 fs_manager.cc:359] Metadata directory not provided
I20260811 02:52:04.585475 15835 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 02:52:04.585717 15835 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 02:52:04.587873 15835 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786416707705494-12843-0/minicluster-data/ts-0/data/instance:
uuid: "f9a104d7ad464393bccc51d4400889b5"
format_stamp: "Formatted at 2026-08-11 02:52:04 on dist-test-slave-9gcw"
server_key: "75b6173b7b91bf1f796700993631339e"
server_key_iv: "6cb2c0b5496d19c326531fd2246334e8"
server_key_version: "encryptionkey@0"
I20260811 02:52:04.588475 15835 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786416707705494-12843-0/minicluster-data/ts-0/wal/instance:
uuid: "f9a104d7ad464393bccc51d4400889b5"
format_stamp: "Formatted at 2026-08-11 02:52:04 on dist-test-slave-9gcw"
server_key: "75b6173b7b91bf1f796700993631339e"
server_key_iv: "6cb2c0b5496d19c326531fd2246334e8"
server_key_version: "encryptionkey@0"
I20260811 02:52:04.592417 15835 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.001s
I20260811 02:52:04.594995 15851 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 02:52:04.596199 15835 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260811 02:52:04.596369 15835 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786416707705494-12843-0/minicluster-data/ts-0/data,/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786416707705494-12843-0/minicluster-data/ts-0/wal
uuid: "f9a104d7ad464393bccc51d4400889b5"
format_stamp: "Formatted at 2026-08-11 02:52:04 on dist-test-slave-9gcw"
server_key: "75b6173b7b91bf1f796700993631339e"
server_key_iv: "6cb2c0b5496d19c326531fd2246334e8"
server_key_version: "encryptionkey@0"
I20260811 02:52:04.596494 15835 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786416707705494-12843-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786416707705494-12843-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786416707705494-12843-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 02:52:04.610121 15835 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 02:52:04.613415 15835 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 02:52:04.613656 15835 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 02:52:04.614312 15835 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 02:52:04.615235 15835 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 02:52:04.615312 15835 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 02:52:04.615427 15835 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 02:52:04.615479 15835 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 02:52:04.624940 15835 rpc_server.cc:307] RPC server started. Bound to: 127.12.138.193:40263
I20260811 02:52:04.624963 15987 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.138.193:40263 every 8 connection(s)
I20260811 02:52:04.626017 15835 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786416707705494-12843-0/minicluster-data/ts-0/data/info.pb
I20260811 02:52:04.631951 12843 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDawEwS/build/debug/bin/kudu as pid 15835
I20260811 02:52:04.632077 12843 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786416707705494-12843-0/minicluster-data/ts-0/wal/instance
I20260811 02:52:04.632381 12843 external_mini_cluster.cc:1428] Setting key 5f9c3d1151bb9535534d2ab31c1b19b4
Aug 11 02:52:04 dist-test-slave-9gcw krb5kdc[15707](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416724, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.138.193@KRBTEST.COM for kudu/127.12.138.254@KRBTEST.COM
I20260811 02:52:04.642019 15812 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:52:04.627982 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.138.193:44327 (local address 127.12.138.254:33819)
0811 02:52:04.628133 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0811 02:52:04.628137 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 02:52:04.629069 (+ 932us) server_negotiation.cc:407] Connection header received
0811 02:52:04.630568 (+ 1499us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:52:04.630572 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:52:04.630623 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:52:04.630716 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:52:04.632221 (+ 1505us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:52:04.632891 (+ 670us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:52:04.634499 (+ 1608us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:52:04.634918 (+ 419us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:52:04.635053 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:52:04.637888 (+ 2835us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:52:04.637891 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:52:04.637893 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:52:04.637917 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:52:04.639485 (+ 1568us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:52:04.640207 (+ 722us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:52:04.640210 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:52:04.640211 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:52:04.640291 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:52:04.640728 (+ 437us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:52:04.640731 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:52:04.640732 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:52:04.640926 (+ 194us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:52:04.641002 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0811 02:52:04.641757 (+ 755us) server_negotiation.cc:299] Negotiation successful
0811 02:52:04.641887 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260811 02:52:04.643360 15990 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:52:04.628387 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.138.254:33819 (local address 127.12.138.193:44327)
0811 02:52:04.628853 (+ 466us) negotiation.cc:107] Waiting for socket to connect
0811 02:52:04.628898 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0811 02:52:04.630264 (+ 1366us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:52:04.630934 (+ 670us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:52:04.630941 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:52:04.631328 (+ 387us) client_negotiation.cc:189] Negotiated authn=SASL
0811 02:52:04.632038 (+ 710us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:52:04.632064 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:52:04.633072 (+ 1008us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:52:04.633075 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:52:04.634366 (+ 1291us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:52:04.634377 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:52:04.635101 (+ 724us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:52:04.635105 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:52:04.635293 (+ 188us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:52:04.636028 (+ 735us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 02:52:04.636047 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0811 02:52:04.637688 (+ 1641us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 02:52:04.639635 (+ 1947us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:52:04.639644 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:52:04.639665 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:52:04.640088 (+ 423us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:52:04.640389 (+ 301us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:52:04.640393 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:52:04.640396 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:52:04.640622 (+ 226us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:52:04.641040 (+ 418us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 02:52:04.641050 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 02:52:04.641474 (+ 424us) client_negotiation.cc:769] Sending connection context
0811 02:52:04.641703 (+ 229us) client_negotiation.cc:240] Negotiation successful
0811 02:52:04.642098 (+ 395us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":258,"thread_start_us":110,"threads_started":1}
WARNING: no policy specified for kudu/127.12.138.194@KRBTEST.COM; defaulting to no policy
I20260811 02:52:04.644747 15988 heartbeater.cc:347] Connected to a master server at 127.12.138.254:33819
I20260811 02:52:04.645049 15988 heartbeater.cc:464] Registering TS with master...
I20260811 02:52:04.645651 15988 heartbeater.cc:511] Master 127.12.138.254:33819 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.138.194@KRBTEST.COM" created.
I20260811 02:52:04.647501 15763 ts_manager.cc:194] Registered new tserver with Master: f9a104d7ad464393bccc51d4400889b5 (127.12.138.193:40263)
I20260811 02:52:04.648737 15763 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.12.138.193@KRBTEST.COM'} at 127.12.138.193:44327
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.138.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786416707705494-12843-0/krb5kdc/kudu_127.12.138.194.keytab.
Entry for principal kudu/127.12.138.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786416707705494-12843-0/krb5kdc/kudu_127.12.138.194.keytab.
I20260811 02:52:04.665594 12843 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDawEwS/build/debug/bin/kudu
/tmp/dist-test-taskDawEwS/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786416707705494-12843-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786416707705494-12843-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786416707705494-12843-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786416707705494-12843-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.138.194:0
--local_ip_for_outbound_sockets=127.12.138.194
--webserver_interface=127.12.138.194
--webserver_port=0
--tserver_master_addrs=127.12.138.254:33819
--keytab_file=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786416707705494-12843-0/krb5kdc/kudu_127.12.138.194.keytab
--principal=kudu/127.12.138.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.138.212:33935
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786416707705494-12843-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786416707705494-12843-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786416707705494-12843-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 02:52:04.797444 15993 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 02:52:04.797770 15993 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 02:52:04.797823 15993 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 02:52:04.802572 15993 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 02:52:04.802660 15993 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 02:52:04.802732 15993 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.138.194
I20260811 02:52:04.808494 15993 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.138.212:33935
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786416707705494-12843-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786416707705494-12843-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786416707705494-12843-0/krb5kdc/kudu_127.12.138.194.keytab
--principal=kudu/127.12.138.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.138.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786416707705494-12843-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.12.138.194
--webserver_port=0
--tserver_master_addrs=127.12.138.254:33819
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15993
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.138.194
--log_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786416707705494-12843-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 02:09:49 UTC on ccycloud.quasar-fsnqqc.root.comops.site
build id 4579570
I20260811 02:52:04.809669 15993 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 02:52:04.810726 15993 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 02:52:04.821403 15998 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:52:04.821419 16001 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:52:04.821435 15999 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:52:04.821607 15993 server_base.cc:1034] running on GCE node
I20260811 02:52:04.822273 15993 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 02:52:04.823043 15993 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 02:52:04.824273 15993 hybrid_clock.cc:648] HybridClock initialized: now 1786416724824255 us; error 33 us; skew 500 ppm
Aug 11 02:52:04 dist-test-slave-9gcw krb5kdc[15707](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416724, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.138.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 02:52:04.827457 15993 init.cc:377] Logged in from keytab as kudu/127.12.138.194@KRBTEST.COM (short username kudu)
I20260811 02:52:04.828478 15993 webserver.cc:460] Webserver started at http://127.12.138.194:34521/ using document root <none> and password file <none>
I20260811 02:52:04.829126 15993 fs_manager.cc:359] Metadata directory not provided
I20260811 02:52:04.829298 15993 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 02:52:04.829588 15993 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 02:52:04.832015 15993 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786416707705494-12843-0/minicluster-data/ts-1/data/instance:
uuid: "fa6885fcf6764757a07b201e56a8b198"
format_stamp: "Formatted at 2026-08-11 02:52:04 on dist-test-slave-9gcw"
server_key: "f225572a1bac593f4d7cf16b4d75f2b8"
server_key_iv: "61b6b25a0d08f0f7b6bf707bf1f2a9b1"
server_key_version: "encryptionkey@0"
I20260811 02:52:04.832700 15993 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786416707705494-12843-0/minicluster-data/ts-1/wal/instance:
uuid: "fa6885fcf6764757a07b201e56a8b198"
format_stamp: "Formatted at 2026-08-11 02:52:04 on dist-test-slave-9gcw"
server_key: "f225572a1bac593f4d7cf16b4d75f2b8"
server_key_iv: "61b6b25a0d08f0f7b6bf707bf1f2a9b1"
server_key_version: "encryptionkey@0"
I20260811 02:52:04.836861 15993 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.002s
I20260811 02:52:04.839407 16009 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 02:52:04.840646 15993 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260811 02:52:04.840804 15993 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786416707705494-12843-0/minicluster-data/ts-1/data,/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786416707705494-12843-0/minicluster-data/ts-1/wal
uuid: "fa6885fcf6764757a07b201e56a8b198"
format_stamp: "Formatted at 2026-08-11 02:52:04 on dist-test-slave-9gcw"
server_key: "f225572a1bac593f4d7cf16b4d75f2b8"
server_key_iv: "61b6b25a0d08f0f7b6bf707bf1f2a9b1"
server_key_version: "encryptionkey@0"
I20260811 02:52:04.840926 15993 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786416707705494-12843-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786416707705494-12843-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786416707705494-12843-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 02:52:04.861313 15993 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 02:52:04.864704 15993 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 02:52:04.864944 15993 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 02:52:04.865628 15993 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 02:52:04.866534 15993 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 02:52:04.866611 15993 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 02:52:04.866686 15993 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 02:52:04.866732 15993 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 02:52:04.876621 15993 rpc_server.cc:307] RPC server started. Bound to: 127.12.138.194:41249
I20260811 02:52:04.876643 16146 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.138.194:41249 every 8 connection(s)
I20260811 02:52:04.877736 15993 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786416707705494-12843-0/minicluster-data/ts-1/data/info.pb
I20260811 02:52:04.883953 12843 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDawEwS/build/debug/bin/kudu as pid 15993
I20260811 02:52:04.884089 12843 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786416707705494-12843-0/minicluster-data/ts-1/wal/instance
I20260811 02:52:04.884449 12843 external_mini_cluster.cc:1428] Setting key d80f7d00318673156756db41675fd892
Aug 11 02:52:04 dist-test-slave-9gcw krb5kdc[15707](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416724, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.138.194@KRBTEST.COM for kudu/127.12.138.254@KRBTEST.COM
I20260811 02:52:04.893371 15812 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:52:04.879723 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.138.194:59023 (local address 127.12.138.254:33819)
0811 02:52:04.879882 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0811 02:52:04.879886 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 02:52:04.880767 (+ 881us) server_negotiation.cc:407] Connection header received
0811 02:52:04.882158 (+ 1391us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:52:04.882161 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:52:04.882211 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:52:04.882289 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:52:04.883522 (+ 1233us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:52:04.884073 (+ 551us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:52:04.885760 (+ 1687us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:52:04.886063 (+ 303us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:52:04.886180 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:52:04.888977 (+ 2797us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:52:04.888984 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:52:04.888988 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:52:04.889022 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:52:04.890747 (+ 1725us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:52:04.891510 (+ 763us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:52:04.891516 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:52:04.891522 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:52:04.891599 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:52:04.892021 (+ 422us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:52:04.892024 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:52:04.892025 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:52:04.892210 (+ 185us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:52:04.892285 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0811 02:52:04.893052 (+ 767us) server_negotiation.cc:299] Negotiation successful
0811 02:52:04.893221 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260811 02:52:04.894701 16151 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:52:04.880122 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.138.254:33819 (local address 127.12.138.194:59023)
0811 02:52:04.880597 (+ 475us) negotiation.cc:107] Waiting for socket to connect
0811 02:52:04.880640 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0811 02:52:04.881942 (+ 1302us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:52:04.882464 (+ 522us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:52:04.882471 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:52:04.882868 (+ 397us) client_negotiation.cc:189] Negotiated authn=SASL
0811 02:52:04.883374 (+ 506us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:52:04.883387 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:52:04.884252 (+ 865us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:52:04.884255 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:52:04.885624 (+ 1369us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:52:04.885634 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:52:04.886227 (+ 593us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:52:04.886231 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:52:04.886421 (+ 190us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:52:04.887094 (+ 673us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 02:52:04.887111 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 02:52:04.888802 (+ 1691us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 02:52:04.890877 (+ 2075us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:52:04.890887 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:52:04.890910 (+ 23us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:52:04.891386 (+ 476us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:52:04.891702 (+ 316us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:52:04.891707 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:52:04.891711 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:52:04.891918 (+ 207us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:52:04.892335 (+ 417us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 02:52:04.892344 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 02:52:04.892748 (+ 404us) client_negotiation.cc:769] Sending connection context
0811 02:52:04.893009 (+ 261us) client_negotiation.cc:240] Negotiation successful
0811 02:52:04.893478 (+ 469us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":284,"thread_start_us":134,"threads_started":1}
I20260811 02:52:04.896050 16147 heartbeater.cc:347] Connected to a master server at 127.12.138.254:33819
WARNING: no policy specified for kudu/127.12.138.195@KRBTEST.COM; defaulting to no policy
I20260811 02:52:04.896363 16147 heartbeater.cc:464] Registering TS with master...
I20260811 02:52:04.896910 16147 heartbeater.cc:511] Master 127.12.138.254:33819 requested a full tablet report, sending...
I20260811 02:52:04.898093 15763 ts_manager.cc:194] Registered new tserver with Master: fa6885fcf6764757a07b201e56a8b198 (127.12.138.194:41249)
I20260811 02:52:04.898748 15763 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.12.138.194@KRBTEST.COM'} at 127.12.138.194:59023
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.138.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.138.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786416707705494-12843-0/krb5kdc/kudu_127.12.138.195.keytab.
Entry for principal kudu/127.12.138.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786416707705494-12843-0/krb5kdc/kudu_127.12.138.195.keytab.
I20260811 02:52:04.917879 12843 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDawEwS/build/debug/bin/kudu
/tmp/dist-test-taskDawEwS/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786416707705494-12843-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786416707705494-12843-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786416707705494-12843-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786416707705494-12843-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.138.195:0
--local_ip_for_outbound_sockets=127.12.138.195
--webserver_interface=127.12.138.195
--webserver_port=0
--tserver_master_addrs=127.12.138.254:33819
--keytab_file=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786416707705494-12843-0/krb5kdc/kudu_127.12.138.195.keytab
--principal=kudu/127.12.138.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.138.212:33935
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786416707705494-12843-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786416707705494-12843-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786416707705494-12843-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 02:52:05.041428 16154 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 02:52:05.041761 16154 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 02:52:05.041869 16154 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 02:52:05.046147 16154 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 02:52:05.046236 16154 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 02:52:05.046324 16154 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.138.195
I20260811 02:52:05.051844 16154 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.138.212:33935
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786416707705494-12843-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786416707705494-12843-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786416707705494-12843-0/krb5kdc/kudu_127.12.138.195.keytab
--principal=kudu/127.12.138.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.138.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786416707705494-12843-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.12.138.195
--webserver_port=0
--tserver_master_addrs=127.12.138.254:33819
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16154
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.138.195
--log_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786416707705494-12843-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 02:09:49 UTC on ccycloud.quasar-fsnqqc.root.comops.site
build id 4579570
I20260811 02:52:05.053026 16154 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 02:52:05.054179 16154 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 02:52:05.064929 16165 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:52:05.064945 16162 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:52:05.064939 16163 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:52:05.065187 16154 server_base.cc:1034] running on GCE node
I20260811 02:52:05.065719 16154 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 02:52:05.066444 16154 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 02:52:05.067619 16154 hybrid_clock.cc:648] HybridClock initialized: now 1786416725067597 us; error 34 us; skew 500 ppm
Aug 11 02:52:05 dist-test-slave-9gcw krb5kdc[15707](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416725, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.138.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 02:52:05.070868 16154 init.cc:377] Logged in from keytab as kudu/127.12.138.195@KRBTEST.COM (short username kudu)
I20260811 02:52:05.071910 16154 webserver.cc:460] Webserver started at http://127.12.138.195:41809/ using document root <none> and password file <none>
I20260811 02:52:05.072477 16154 fs_manager.cc:359] Metadata directory not provided
I20260811 02:52:05.072540 16154 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 02:52:05.072721 16154 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 02:52:05.075016 16154 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786416707705494-12843-0/minicluster-data/ts-2/data/instance:
uuid: "16038d8cae7a4a1eaf55d1aedd6ae15a"
format_stamp: "Formatted at 2026-08-11 02:52:05 on dist-test-slave-9gcw"
server_key: "8ec83f11e0bdba5b72e293969a8f45b6"
server_key_iv: "150717300be8c7f38531b717f06c429f"
server_key_version: "encryptionkey@0"
I20260811 02:52:05.075608 16154 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786416707705494-12843-0/minicluster-data/ts-2/wal/instance:
uuid: "16038d8cae7a4a1eaf55d1aedd6ae15a"
format_stamp: "Formatted at 2026-08-11 02:52:05 on dist-test-slave-9gcw"
server_key: "8ec83f11e0bdba5b72e293969a8f45b6"
server_key_iv: "150717300be8c7f38531b717f06c429f"
server_key_version: "encryptionkey@0"
I20260811 02:52:05.079624 16154 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260811 02:52:05.082332 16173 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 02:52:05.084090 16154 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 02:52:05.084221 16154 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786416707705494-12843-0/minicluster-data/ts-2/data,/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786416707705494-12843-0/minicluster-data/ts-2/wal
uuid: "16038d8cae7a4a1eaf55d1aedd6ae15a"
format_stamp: "Formatted at 2026-08-11 02:52:05 on dist-test-slave-9gcw"
server_key: "8ec83f11e0bdba5b72e293969a8f45b6"
server_key_iv: "150717300be8c7f38531b717f06c429f"
server_key_version: "encryptionkey@0"
I20260811 02:52:05.084331 16154 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786416707705494-12843-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786416707705494-12843-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786416707705494-12843-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 02:52:05.102247 16154 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 02:52:05.105693 16154 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 02:52:05.105939 16154 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 02:52:05.106616 16154 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 02:52:05.107514 16154 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 02:52:05.107565 16154 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 02:52:05.107635 16154 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 02:52:05.107668 16154 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 02:52:05.116855 16154 rpc_server.cc:307] RPC server started. Bound to: 127.12.138.195:36739
I20260811 02:52:05.116884 16301 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.138.195:36739 every 8 connection(s)
I20260811 02:52:05.117853 16154 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786416707705494-12843-0/minicluster-data/ts-2/data/info.pb
I20260811 02:52:05.126142 12843 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDawEwS/build/debug/bin/kudu as pid 16154
I20260811 02:52:05.126289 12843 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786416707705494-12843-0/minicluster-data/ts-2/wal/instance
I20260811 02:52:05.126611 12843 external_mini_cluster.cc:1428] Setting key a4e2153bca97907158c8b9bcb0a56f9c
Aug 11 02:52:05 dist-test-slave-9gcw krb5kdc[15707](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416725, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.138.195@KRBTEST.COM for kudu/127.12.138.254@KRBTEST.COM
I20260811 02:52:05.132118 15812 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:52:05.119692 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.138.195:46213 (local address 127.12.138.254:33819)
0811 02:52:05.119840 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0811 02:52:05.119844 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 02:52:05.120708 (+ 864us) server_negotiation.cc:407] Connection header received
0811 02:52:05.122061 (+ 1353us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:52:05.122063 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:52:05.122116 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:52:05.122193 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:52:05.123365 (+ 1172us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:52:05.123909 (+ 544us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:52:05.125036 (+ 1127us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:52:05.125365 (+ 329us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:52:05.125454 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:52:05.128206 (+ 2752us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:52:05.128209 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:52:05.128211 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:52:05.128237 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:52:05.129983 (+ 1746us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:52:05.130476 (+ 493us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:52:05.130478 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:52:05.130480 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:52:05.130556 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:52:05.130983 (+ 427us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:52:05.130985 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:52:05.130987 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:52:05.131165 (+ 178us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:52:05.131250 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0811 02:52:05.131860 (+ 610us) server_negotiation.cc:299] Negotiation successful
0811 02:52:05.131997 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260811 02:52:05.133037 16306 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:52:05.120053 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.138.254:33819 (local address 127.12.138.195:46213)
0811 02:52:05.120540 (+ 487us) negotiation.cc:107] Waiting for socket to connect
0811 02:52:05.120582 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0811 02:52:05.121844 (+ 1262us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:52:05.122373 (+ 529us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:52:05.122380 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:52:05.122733 (+ 353us) client_negotiation.cc:189] Negotiated authn=SASL
0811 02:52:05.123208 (+ 475us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:52:05.123221 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:52:05.124017 (+ 796us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:52:05.124020 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:52:05.124915 (+ 895us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:52:05.124924 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:52:05.125498 (+ 574us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:52:05.125500 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:52:05.125646 (+ 146us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:52:05.126346 (+ 700us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 02:52:05.126370 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0811 02:52:05.128018 (+ 1648us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 02:52:05.130085 (+ 2067us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:52:05.130090 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:52:05.130102 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:52:05.130388 (+ 286us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:52:05.130678 (+ 290us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:52:05.130681 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:52:05.130682 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:52:05.130898 (+ 216us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:52:05.131264 (+ 366us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 02:52:05.131269 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 02:52:05.131560 (+ 291us) client_negotiation.cc:769] Sending connection context
0811 02:52:05.131756 (+ 196us) client_negotiation.cc:240] Negotiation successful
0811 02:52:05.132083 (+ 327us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":291,"thread_start_us":125,"threads_started":1}
I20260811 02:52:05.134186 16302 heartbeater.cc:347] Connected to a master server at 127.12.138.254:33819
I20260811 02:52:05.134428 16302 heartbeater.cc:464] Registering TS with master...
I20260811 02:52:05.134925 16302 heartbeater.cc:511] Master 127.12.138.254:33819 requested a full tablet report, sending...
I20260811 02:52:05.135939 15763 ts_manager.cc:194] Registered new tserver with Master: 16038d8cae7a4a1eaf55d1aedd6ae15a (127.12.138.195:36739)
I20260811 02:52:05.136464 15763 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.12.138.195@KRBTEST.COM'} at 127.12.138.195:46213
I20260811 02:52:05.140377 12843 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260811 02:52:05.155498 15812 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 02:52:05.153994 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42788 (local address 127.12.138.254:33819)
0811 02:52:05.154133 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0811 02:52:05.154137 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 02:52:05.154300 (+ 163us) server_negotiation.cc:407] Connection header received
0811 02:52:05.154479 (+ 179us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:52:05.154482 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:52:05.154531 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:52:05.154703 (+ 172us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:52:05.155383 (+ 680us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:42788: BlockingRecv error: recv got EOF from 127.0.0.1:42788 (error 108)
Metrics: {"server-negotiator.queue_time_us":55}
I20260811 02:52:05.156816 12843 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDawEwS/build/debug/bin/kudu with pid 15835
I20260811 02:52:05.169812 12843 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDawEwS/build/debug/bin/kudu with pid 15993
I20260811 02:52:05.177769 12843 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDawEwS/build/debug/bin/kudu with pid 16154
I20260811 02:52:05.188395 12843 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDawEwS/build/debug/bin/kudu with pid 15723
2026-08-11T02:52:05Z chronyd exiting
[ OK ] SecurityITest.TestNoKerberosCredentials (2468 ms)
[ RUN ] SecurityITest.SaslPlainFallback
Loading random data
Initializing database '/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786416707705494-12843-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 02:52:05 dist-test-slave-9gcw krb5kdc[16317](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 02:52:05 dist-test-slave-9gcw krb5kdc[16317](info): set up 2 sockets
Aug 11 02:52:05 dist-test-slave-9gcw krb5kdc[16317](info): commencing operation
krb5kdc: starting...
W20260811 02:52:06.387208 12843 mini_kdc.cc:121] Time spent starting KDC: real 1.172s user 0.002s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786416707705494-12843-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786416707705494-12843-0/krb5kdc/test-user.keytab.
Aug 11 02:52:06 dist-test-slave-9gcw krb5kdc[16317](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416726, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T02:52:06Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T02:52:06Z Disabled control of system clock
WARNING: no policy specified for kudu/127.12.138.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.138.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.138.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786416707705494-12843-0/krb5kdc/kudu_127.12.138.254.keytab.
Entry for principal kudu/127.12.138.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786416707705494-12843-0/krb5kdc/kudu_127.12.138.254.keytab.
I20260811 02:52:06.520184 12843 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDawEwS/build/debug/bin/kudu
/tmp/dist-test-taskDawEwS/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786416707705494-12843-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786416707705494-12843-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786416707705494-12843-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786416707705494-12843-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.138.254:33383
--webserver_interface=127.12.138.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786416707705494-12843-0/krb5kdc/kudu_127.12.138.254.keytab
--principal=kudu/127.12.138.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.138.212:43397
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.12.138.254:33383
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc-authentication=optional
--user-acl=* with env {KRB5CCNAME=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786416707705494-12843-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786416707705494-12843-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786416707705494-12843-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 02:52:06.633736 16332 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 02:52:06.634059 16332 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 02:52:06.634142 16332 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 02:52:06.638398 16332 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 02:52:06.638490 16332 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 02:52:06.638528 16332 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 02:52:06.638569 16332 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 02:52:06.638602 16332 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 02:52:06.644033 16332 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.138.212:43397
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786416707705494-12843-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786416707705494-12843-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.12.138.254:33383
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786416707705494-12843-0/krb5kdc/kudu_127.12.138.254.keytab
--principal=kudu/127.12.138.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.138.254:33383
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786416707705494-12843-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.12.138.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16332
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786416707705494-12843-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 02:09:49 UTC on ccycloud.quasar-fsnqqc.root.comops.site
build id 4579570
I20260811 02:52:06.645216 16332 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 02:52:06.646382 16332 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 02:52:06.655258 16338 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:52:06.655274 16337 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:52:06.655274 16341 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:52:06.655397 16332 server_base.cc:1034] running on GCE node
I20260811 02:52:06.656199 16332 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 02:52:06.657375 16332 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 02:52:06.658551 16332 hybrid_clock.cc:648] HybridClock initialized: now 1786416726658531 us; error 33 us; skew 500 ppm
Aug 11 02:52:06 dist-test-slave-9gcw krb5kdc[16317](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416726, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.138.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 02:52:06.666108 16332 init.cc:377] Logged in from keytab as kudu/127.12.138.254@KRBTEST.COM (short username kudu)
I20260811 02:52:06.667138 16332 webserver.cc:460] Webserver started at http://127.12.138.254:45411/ using document root <none> and password file <none>
I20260811 02:52:06.667742 16332 fs_manager.cc:359] Metadata directory not provided
I20260811 02:52:06.667806 16332 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 02:52:06.668088 16332 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 02:52:06.670267 16332 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786416707705494-12843-0/minicluster-data/master-0/data/instance:
uuid: "46769e1e096e4143b8e9e10b26454b15"
format_stamp: "Formatted at 2026-08-11 02:52:06 on dist-test-slave-9gcw"
server_key: "9170d3582e43698bbbd8e8377d281b4e"
server_key_iv: "37fe4bdd9764d33a2cafb45cd1770921"
server_key_version: "encryptionkey@0"
I20260811 02:52:06.670843 16332 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786416707705494-12843-0/minicluster-data/master-0/wal/instance:
uuid: "46769e1e096e4143b8e9e10b26454b15"
format_stamp: "Formatted at 2026-08-11 02:52:06 on dist-test-slave-9gcw"
server_key: "9170d3582e43698bbbd8e8377d281b4e"
server_key_iv: "37fe4bdd9764d33a2cafb45cd1770921"
server_key_version: "encryptionkey@0"
I20260811 02:52:06.674795 16332 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260811 02:52:06.677350 16352 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 02:52:06.678748 16332 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260811 02:52:06.678881 16332 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786416707705494-12843-0/minicluster-data/master-0/data,/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786416707705494-12843-0/minicluster-data/master-0/wal
uuid: "46769e1e096e4143b8e9e10b26454b15"
format_stamp: "Formatted at 2026-08-11 02:52:06 on dist-test-slave-9gcw"
server_key: "9170d3582e43698bbbd8e8377d281b4e"
server_key_iv: "37fe4bdd9764d33a2cafb45cd1770921"
server_key_version: "encryptionkey@0"
I20260811 02:52:06.678997 16332 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786416707705494-12843-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786416707705494-12843-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786416707705494-12843-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 02:52:06.703666 16332 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 02:52:06.707417 16332 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 02:52:06.707646 16332 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 02:52:06.714957 16332 rpc_server.cc:307] RPC server started. Bound to: 127.12.138.254:33383
I20260811 02:52:06.715255 16413 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.138.254:33383 every 8 connection(s)
I20260811 02:52:06.716050 16332 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786416707705494-12843-0/minicluster-data/master-0/data/info.pb
I20260811 02:52:06.717391 12843 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDawEwS/build/debug/bin/kudu as pid 16332
I20260811 02:52:06.717545 12843 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786416707705494-12843-0/minicluster-data/master-0/wal/instance
I20260811 02:52:06.717859 12843 external_mini_cluster.cc:1428] Setting key bb5af972046943a191f2c21d57023164
I20260811 02:52:06.719560 16414 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 02:52:06.726651 16414 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 46769e1e096e4143b8e9e10b26454b15: Bootstrap starting.
Aug 11 02:52:06 dist-test-slave-9gcw krb5kdc[16317](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416726, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.12.138.254@KRBTEST.COM
I20260811 02:52:06.729486 16414 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 46769e1e096e4143b8e9e10b26454b15: Neither blocks nor log segments found. Creating new log.
I20260811 02:52:06.730486 16414 log.cc:824] T 00000000000000000000000000000000 P 46769e1e096e4143b8e9e10b26454b15: Log is configured to *not* fsync() on all Append() calls
I20260811 02:52:06.732803 16414 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 46769e1e096e4143b8e9e10b26454b15: No bootstrap required, opened a new log
I20260811 02:52:06.734019 16417 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:52:06.719344 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32908 (local address 127.12.138.254:33383)
0811 02:52:06.719798 (+ 454us) server_negotiation.cc:206] Beginning negotiation
0811 02:52:06.719819 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0811 02:52:06.719861 (+ 42us) server_negotiation.cc:407] Connection header received
0811 02:52:06.720705 (+ 844us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:52:06.720728 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:52:06.721204 (+ 476us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:52:06.721591 (+ 387us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:52:06.722516 (+ 925us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:52:06.724009 (+ 1493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:52:06.724938 (+ 929us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:52:06.725708 (+ 770us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:52:06.725893 (+ 185us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:52:06.728717 (+ 2824us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:52:06.728723 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:52:06.728739 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:52:06.728773 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:52:06.731005 (+ 2232us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:52:06.731508 (+ 503us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:52:06.731514 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:52:06.731526 (+ 12us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:52:06.731642 (+ 116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:52:06.731967 (+ 325us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:52:06.731977 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:52:06.731978 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:52:06.732383 (+ 405us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:52:06.732585 (+ 202us) server_negotiation.cc:1035] Waiting for connection context
0811 02:52:06.732918 (+ 333us) server_negotiation.cc:299] Negotiation successful
0811 02:52:06.733280 (+ 362us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":287,"thread_start_us":129,"threads_started":1}
I20260811 02:52:06.737370 12843 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 02:52:06.737721 16414 raft_consensus.cc:348] T 00000000000000000000000000000000 P 46769e1e096e4143b8e9e10b26454b15 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "46769e1e096e4143b8e9e10b26454b15" member_type: VOTER last_known_addr { host: "127.12.138.254" port: 33383 } }
I20260811 02:52:06.738009 16414 raft_consensus.cc:374] T 00000000000000000000000000000000 P 46769e1e096e4143b8e9e10b26454b15 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 02:52:06.738112 16414 raft_consensus.cc:729] T 00000000000000000000000000000000 P 46769e1e096e4143b8e9e10b26454b15 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 46769e1e096e4143b8e9e10b26454b15, State: Initialized, Role: FOLLOWER
I20260811 02:52:06.738634 16414 consensus_queue.cc:260] T 00000000000000000000000000000000 P 46769e1e096e4143b8e9e10b26454b15 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "46769e1e096e4143b8e9e10b26454b15" member_type: VOTER last_known_addr { host: "127.12.138.254" port: 33383 } }
I20260811 02:52:06.738801 16414 raft_consensus.cc:388] T 00000000000000000000000000000000 P 46769e1e096e4143b8e9e10b26454b15 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 02:52:06.738899 16414 raft_consensus.cc:482] T 00000000000000000000000000000000 P 46769e1e096e4143b8e9e10b26454b15 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 02:52:06.739017 16414 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 46769e1e096e4143b8e9e10b26454b15 [term 0 FOLLOWER]: Advancing to term 1
I20260811 02:52:06.740103 16414 raft_consensus.cc:504] T 00000000000000000000000000000000 P 46769e1e096e4143b8e9e10b26454b15 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "46769e1e096e4143b8e9e10b26454b15" member_type: VOTER last_known_addr { host: "127.12.138.254" port: 33383 } }
I20260811 02:52:06.740491 16414 leader_election.cc:302] T 00000000000000000000000000000000 P 46769e1e096e4143b8e9e10b26454b15 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 46769e1e096e4143b8e9e10b26454b15; no voters:
I20260811 02:52:06.740895 16414 leader_election.cc:288] T 00000000000000000000000000000000 P 46769e1e096e4143b8e9e10b26454b15 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 02:52:06.741034 16422 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 46769e1e096e4143b8e9e10b26454b15 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 02:52:06.741317 16422 raft_consensus.cc:686] T 00000000000000000000000000000000 P 46769e1e096e4143b8e9e10b26454b15 [term 1 LEADER]: Becoming Leader. State: Replica: 46769e1e096e4143b8e9e10b26454b15, State: Running, Role: LEADER
I20260811 02:52:06.741719 16422 consensus_queue.cc:237] T 00000000000000000000000000000000 P 46769e1e096e4143b8e9e10b26454b15 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "46769e1e096e4143b8e9e10b26454b15" member_type: VOTER last_known_addr { host: "127.12.138.254" port: 33383 } }
I20260811 02:52:06.742341 16414 sys_catalog.cc:564] T 00000000000000000000000000000000 P 46769e1e096e4143b8e9e10b26454b15 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 02:52:06.743034 16422 sys_catalog.cc:455] T 00000000000000000000000000000000 P 46769e1e096e4143b8e9e10b26454b15 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 46769e1e096e4143b8e9e10b26454b15. Latest consensus state: current_term: 1 leader_uuid: "46769e1e096e4143b8e9e10b26454b15" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "46769e1e096e4143b8e9e10b26454b15" member_type: VOTER last_known_addr { host: "127.12.138.254" port: 33383 } } }
I20260811 02:52:06.743266 16422 sys_catalog.cc:458] T 00000000000000000000000000000000 P 46769e1e096e4143b8e9e10b26454b15 [sys.catalog]: This master's current role is: LEADER
I20260811 02:52:06.743542 16423 sys_catalog.cc:455] T 00000000000000000000000000000000 P 46769e1e096e4143b8e9e10b26454b15 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "46769e1e096e4143b8e9e10b26454b15" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "46769e1e096e4143b8e9e10b26454b15" member_type: VOTER last_known_addr { host: "127.12.138.254" port: 33383 } } }
I20260811 02:52:06.743755 16423 sys_catalog.cc:458] T 00000000000000000000000000000000 P 46769e1e096e4143b8e9e10b26454b15 [sys.catalog]: This master's current role is: LEADER
I20260811 02:52:06.743779 16431 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 02:52:06.747706 16431 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 02:52:06.754595 16431 catalog_manager.cc:1347] Generated new cluster ID: e4555d3549f343b298d3e9642ee5d511
I20260811 02:52:06.754670 16431 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 02:52:06.760182 16417 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:52:06.754965 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32924 (local address 127.12.138.254:33383)
0811 02:52:06.755097 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0811 02:52:06.755105 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 02:52:06.755210 (+ 105us) server_negotiation.cc:407] Connection header received
0811 02:52:06.755378 (+ 168us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:52:06.755381 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:52:06.755440 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:52:06.755529 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:52:06.756125 (+ 596us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:52:06.756752 (+ 627us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:52:06.757774 (+ 1022us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:52:06.758033 (+ 259us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:52:06.758167 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:52:06.759151 (+ 984us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:52:06.759155 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:52:06.759158 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 02:52:06.759614 (+ 456us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:52:06.759655 (+ 41us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 02:52:06.759674 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:52:06.759787 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0811 02:52:06.759907 (+ 120us) server_negotiation.cc:299] Negotiation successful
0811 02:52:06.759977 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":48}
I20260811 02:52:06.781016 16431 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 02:52:06.782136 16431 catalog_manager.cc:1504] Loading token signing keys...
I20260811 02:52:06.791028 16431 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 46769e1e096e4143b8e9e10b26454b15: Generated new TSK 0
I20260811 02:52:06.791872 16431 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 02:52:06.801474 12843 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDawEwS/build/debug/bin/kudu with pid 16332
2026-08-11T02:52:06Z chronyd exiting
[ OK ] SecurityITest.SaslPlainFallback (1610 ms)
[ RUN ] SecurityITest.TestUnauthorizedClientKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786416707705494-12843-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 02:52:06 dist-test-slave-9gcw krb5kdc[16453](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 02:52:06 dist-test-slave-9gcw krb5kdc[16453](info): set up 2 sockets
Aug 11 02:52:06 dist-test-slave-9gcw krb5kdc[16453](info): commencing operation
krb5kdc: starting...
W20260811 02:52:07.979669 12843 mini_kdc.cc:121] Time spent starting KDC: real 1.154s user 0.004s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786416707705494-12843-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786416707705494-12843-0/krb5kdc/test-user.keytab.
Aug 11 02:52:08 dist-test-slave-9gcw krb5kdc[16453](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416728, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T02:52:08Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T02:52:08Z Disabled control of system clock
WARNING: no policy specified for kudu/127.12.138.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.138.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.138.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786416707705494-12843-0/krb5kdc/kudu_127.12.138.254.keytab.
Entry for principal kudu/127.12.138.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786416707705494-12843-0/krb5kdc/kudu_127.12.138.254.keytab.
I20260811 02:52:08.129208 12843 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDawEwS/build/debug/bin/kudu
/tmp/dist-test-taskDawEwS/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786416707705494-12843-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786416707705494-12843-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786416707705494-12843-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786416707705494-12843-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.138.254:46489
--webserver_interface=127.12.138.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786416707705494-12843-0/krb5kdc/kudu_127.12.138.254.keytab
--principal=kudu/127.12.138.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.138.212:36249
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.12.138.254:46489
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786416707705494-12843-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786416707705494-12843-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786416707705494-12843-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 02:52:08.248778 16467 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 02:52:08.249066 16467 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 02:52:08.249118 16467 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 02:52:08.253201 16467 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 02:52:08.253268 16467 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 02:52:08.253291 16467 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 02:52:08.253350 16467 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 02:52:08.253392 16467 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 02:52:08.259075 16467 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.138.212:36249
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786416707705494-12843-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786416707705494-12843-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.12.138.254:46489
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786416707705494-12843-0/krb5kdc/kudu_127.12.138.254.keytab
--principal=kudu/127.12.138.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.138.254:46489
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786416707705494-12843-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.12.138.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16467
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786416707705494-12843-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 02:09:49 UTC on ccycloud.quasar-fsnqqc.root.comops.site
build id 4579570
I20260811 02:52:08.260264 16467 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 02:52:08.261431 16467 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 02:52:08.270833 16473 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:52:08.270840 16472 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:52:08.270954 16475 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:52:08.271166 16467 server_base.cc:1034] running on GCE node
I20260811 02:52:08.271847 16467 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 02:52:08.273116 16467 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 02:52:08.274405 16467 hybrid_clock.cc:648] HybridClock initialized: now 1786416728274394 us; error 35 us; skew 500 ppm
Aug 11 02:52:08 dist-test-slave-9gcw krb5kdc[16453](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416728, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.138.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 02:52:08.278060 16467 init.cc:377] Logged in from keytab as kudu/127.12.138.254@KRBTEST.COM (short username kudu)
I20260811 02:52:08.279093 16467 webserver.cc:460] Webserver started at http://127.12.138.254:34779/ using document root <none> and password file <none>
I20260811 02:52:08.279784 16467 fs_manager.cc:359] Metadata directory not provided
I20260811 02:52:08.279892 16467 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 02:52:08.280158 16467 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 02:52:08.282476 16467 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786416707705494-12843-0/minicluster-data/master-0/data/instance:
uuid: "49778fb5a7324e629193ec142237cefd"
format_stamp: "Formatted at 2026-08-11 02:52:08 on dist-test-slave-9gcw"
server_key: "c86a105a06109651f68f481120547fbc"
server_key_iv: "3f837fadf2a879f07e9c0b78940ed133"
server_key_version: "encryptionkey@0"
I20260811 02:52:08.283082 16467 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786416707705494-12843-0/minicluster-data/master-0/wal/instance:
uuid: "49778fb5a7324e629193ec142237cefd"
format_stamp: "Formatted at 2026-08-11 02:52:08 on dist-test-slave-9gcw"
server_key: "c86a105a06109651f68f481120547fbc"
server_key_iv: "3f837fadf2a879f07e9c0b78940ed133"
server_key_version: "encryptionkey@0"
I20260811 02:52:08.287197 16467 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260811 02:52:08.289743 16482 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 02:52:08.290961 16467 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260811 02:52:08.291124 16467 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786416707705494-12843-0/minicluster-data/master-0/data,/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786416707705494-12843-0/minicluster-data/master-0/wal
uuid: "49778fb5a7324e629193ec142237cefd"
format_stamp: "Formatted at 2026-08-11 02:52:08 on dist-test-slave-9gcw"
server_key: "c86a105a06109651f68f481120547fbc"
server_key_iv: "3f837fadf2a879f07e9c0b78940ed133"
server_key_version: "encryptionkey@0"
I20260811 02:52:08.291245 16467 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786416707705494-12843-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786416707705494-12843-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786416707705494-12843-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 02:52:08.309773 16467 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 02:52:08.312861 16467 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 02:52:08.313092 16467 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 02:52:08.320716 16467 rpc_server.cc:307] RPC server started. Bound to: 127.12.138.254:46489
I20260811 02:52:08.321010 16544 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.138.254:46489 every 8 connection(s)
I20260811 02:52:08.321770 16467 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786416707705494-12843-0/minicluster-data/master-0/data/info.pb
I20260811 02:52:08.324640 16545 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 02:52:08.326936 12843 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDawEwS/build/debug/bin/kudu as pid 16467
I20260811 02:52:08.327054 12843 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786416707705494-12843-0/minicluster-data/master-0/wal/instance
I20260811 02:52:08.327359 12843 external_mini_cluster.cc:1428] Setting key e2403a702c3abc7bdca5623b0a7e5596
I20260811 02:52:08.331703 16545 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 49778fb5a7324e629193ec142237cefd: Bootstrap starting.
I20260811 02:52:08.334244 16545 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 49778fb5a7324e629193ec142237cefd: Neither blocks nor log segments found. Creating new log.
I20260811 02:52:08.335175 16545 log.cc:824] T 00000000000000000000000000000000 P 49778fb5a7324e629193ec142237cefd: Log is configured to *not* fsync() on all Append() calls
Aug 11 02:52:08 dist-test-slave-9gcw krb5kdc[16453](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416728, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.12.138.254@KRBTEST.COM
I20260811 02:52:08.337258 16545 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 49778fb5a7324e629193ec142237cefd: No bootstrap required, opened a new log
I20260811 02:52:08.340826 16545 raft_consensus.cc:348] T 00000000000000000000000000000000 P 49778fb5a7324e629193ec142237cefd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "49778fb5a7324e629193ec142237cefd" member_type: VOTER last_known_addr { host: "127.12.138.254" port: 46489 } }
I20260811 02:52:08.341107 16545 raft_consensus.cc:374] T 00000000000000000000000000000000 P 49778fb5a7324e629193ec142237cefd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 02:52:08.341251 16545 raft_consensus.cc:729] T 00000000000000000000000000000000 P 49778fb5a7324e629193ec142237cefd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 49778fb5a7324e629193ec142237cefd, State: Initialized, Role: FOLLOWER
I20260811 02:52:08.341706 16545 consensus_queue.cc:260] T 00000000000000000000000000000000 P 49778fb5a7324e629193ec142237cefd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "49778fb5a7324e629193ec142237cefd" member_type: VOTER last_known_addr { host: "127.12.138.254" port: 46489 } }
I20260811 02:52:08.341841 16545 raft_consensus.cc:388] T 00000000000000000000000000000000 P 49778fb5a7324e629193ec142237cefd [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 02:52:08.341935 16545 raft_consensus.cc:482] T 00000000000000000000000000000000 P 49778fb5a7324e629193ec142237cefd [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 02:52:08.342068 16545 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 49778fb5a7324e629193ec142237cefd [term 0 FOLLOWER]: Advancing to term 1
I20260811 02:52:08.342279 16548 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:52:08.328762 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58362 (local address 127.12.138.254:46489)
0811 02:52:08.329208 (+ 446us) server_negotiation.cc:206] Beginning negotiation
0811 02:52:08.329231 (+ 23us) server_negotiation.cc:399] Waiting for connection header
0811 02:52:08.329277 (+ 46us) server_negotiation.cc:407] Connection header received
0811 02:52:08.329915 (+ 638us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:52:08.329932 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:52:08.330256 (+ 324us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:52:08.330526 (+ 270us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:52:08.331428 (+ 902us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:52:08.332560 (+ 1132us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:52:08.333766 (+ 1206us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:52:08.334245 (+ 479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:52:08.334455 (+ 210us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:52:08.337207 (+ 2752us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:52:08.337214 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:52:08.337229 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:52:08.337256 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:52:08.338957 (+ 1701us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:52:08.339443 (+ 486us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:52:08.339448 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:52:08.339452 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:52:08.339569 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:52:08.339980 (+ 411us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:52:08.339984 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:52:08.339985 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:52:08.340385 (+ 400us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:52:08.340620 (+ 235us) server_negotiation.cc:1035] Waiting for connection context
0811 02:52:08.340877 (+ 257us) server_negotiation.cc:299] Negotiation successful
0811 02:52:08.341264 (+ 387us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":279,"thread_start_us":145,"threads_started":1}
I20260811 02:52:08.343094 16545 raft_consensus.cc:504] T 00000000000000000000000000000000 P 49778fb5a7324e629193ec142237cefd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "49778fb5a7324e629193ec142237cefd" member_type: VOTER last_known_addr { host: "127.12.138.254" port: 46489 } }
I20260811 02:52:08.343441 16545 leader_election.cc:302] T 00000000000000000000000000000000 P 49778fb5a7324e629193ec142237cefd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 49778fb5a7324e629193ec142237cefd; no voters:
I20260811 02:52:08.343752 16545 leader_election.cc:288] T 00000000000000000000000000000000 P 49778fb5a7324e629193ec142237cefd [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 02:52:08.343879 16550 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 49778fb5a7324e629193ec142237cefd [term 1 FOLLOWER]: Leader election won for term 1
I20260811 02:52:08.344125 16550 raft_consensus.cc:686] T 00000000000000000000000000000000 P 49778fb5a7324e629193ec142237cefd [term 1 LEADER]: Becoming Leader. State: Replica: 49778fb5a7324e629193ec142237cefd, State: Running, Role: LEADER
I20260811 02:52:08.344467 16550 consensus_queue.cc:237] T 00000000000000000000000000000000 P 49778fb5a7324e629193ec142237cefd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "49778fb5a7324e629193ec142237cefd" member_type: VOTER last_known_addr { host: "127.12.138.254" port: 46489 } }
I20260811 02:52:08.344998 16545 sys_catalog.cc:564] T 00000000000000000000000000000000 P 49778fb5a7324e629193ec142237cefd [sys.catalog]: configured and running, proceeding with master startup.
I20260811 02:52:08.345674 16550 sys_catalog.cc:455] T 00000000000000000000000000000000 P 49778fb5a7324e629193ec142237cefd [sys.catalog]: SysCatalogTable state changed. Reason: New leader 49778fb5a7324e629193ec142237cefd. Latest consensus state: current_term: 1 leader_uuid: "49778fb5a7324e629193ec142237cefd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "49778fb5a7324e629193ec142237cefd" member_type: VOTER last_known_addr { host: "127.12.138.254" port: 46489 } } }
I20260811 02:52:08.345860 16550 sys_catalog.cc:458] T 00000000000000000000000000000000 P 49778fb5a7324e629193ec142237cefd [sys.catalog]: This master's current role is: LEADER
I20260811 02:52:08.346305 16561 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 02:52:08.346227 16551 sys_catalog.cc:455] T 00000000000000000000000000000000 P 49778fb5a7324e629193ec142237cefd [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "49778fb5a7324e629193ec142237cefd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "49778fb5a7324e629193ec142237cefd" member_type: VOTER last_known_addr { host: "127.12.138.254" port: 46489 } } }
I20260811 02:52:08.346414 16551 sys_catalog.cc:458] T 00000000000000000000000000000000 P 49778fb5a7324e629193ec142237cefd [sys.catalog]: This master's current role is: LEADER
I20260811 02:52:08.349788 16561 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 02:52:08.356521 16561 catalog_manager.cc:1347] Generated new cluster ID: daf2a5284a3c4790972ca028cb72ce27
I20260811 02:52:08.356616 16561 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 02:52:08.377558 16561 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 02:52:08.378969 16561 catalog_manager.cc:1504] Loading token signing keys...
I20260811 02:52:08.384083 16561 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 49778fb5a7324e629193ec142237cefd: Generated new TSK 0
I20260811 02:52:08.384860 16561 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.12.138.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.138.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.138.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786416707705494-12843-0/krb5kdc/kudu_127.12.138.193.keytab.
Entry for principal kudu/127.12.138.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786416707705494-12843-0/krb5kdc/kudu_127.12.138.193.keytab.
I20260811 02:52:08.432650 12843 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDawEwS/build/debug/bin/kudu
/tmp/dist-test-taskDawEwS/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786416707705494-12843-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786416707705494-12843-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786416707705494-12843-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786416707705494-12843-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.138.193:0
--local_ip_for_outbound_sockets=127.12.138.193
--webserver_interface=127.12.138.193
--webserver_port=0
--tserver_master_addrs=127.12.138.254:46489
--keytab_file=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786416707705494-12843-0/krb5kdc/kudu_127.12.138.193.keytab
--principal=kudu/127.12.138.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.138.212:36249
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786416707705494-12843-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786416707705494-12843-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786416707705494-12843-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 02:52:08.551446 16575 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 02:52:08.551723 16575 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 02:52:08.551767 16575 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 02:52:08.555914 16575 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 02:52:08.555981 16575 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 02:52:08.556077 16575 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.138.193
I20260811 02:52:08.561480 16575 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.138.212:36249
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786416707705494-12843-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786416707705494-12843-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786416707705494-12843-0/krb5kdc/kudu_127.12.138.193.keytab
--principal=kudu/127.12.138.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.138.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786416707705494-12843-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.12.138.193
--webserver_port=0
--tserver_master_addrs=127.12.138.254:46489
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16575
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.138.193
--log_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786416707705494-12843-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 02:09:49 UTC on ccycloud.quasar-fsnqqc.root.comops.site
build id 4579570
I20260811 02:52:08.562587 16575 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 02:52:08.563619 16575 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 02:52:08.573438 16581 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:52:08.573410 16583 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:52:08.573601 16580 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:52:08.573635 16575 server_base.cc:1034] running on GCE node
I20260811 02:52:08.574149 16575 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 02:52:08.574738 16575 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 02:52:08.575912 16575 hybrid_clock.cc:648] HybridClock initialized: now 1786416728575880 us; error 45 us; skew 500 ppm
Aug 11 02:52:08 dist-test-slave-9gcw krb5kdc[16453](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416728, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.138.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 02:52:08.578902 16575 init.cc:377] Logged in from keytab as kudu/127.12.138.193@KRBTEST.COM (short username kudu)
I20260811 02:52:08.579903 16575 webserver.cc:460] Webserver started at http://127.12.138.193:33009/ using document root <none> and password file <none>
I20260811 02:52:08.580440 16575 fs_manager.cc:359] Metadata directory not provided
I20260811 02:52:08.580499 16575 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 02:52:08.580667 16575 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 02:52:08.582991 16575 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786416707705494-12843-0/minicluster-data/ts-0/data/instance:
uuid: "dfef187d3b2e432da1643c3bad3ed0e8"
format_stamp: "Formatted at 2026-08-11 02:52:08 on dist-test-slave-9gcw"
server_key: "bf0e4e33ca60cdfea3aa4490886ecf07"
server_key_iv: "2e4d346a52bcca645c2d29107226c17e"
server_key_version: "encryptionkey@0"
I20260811 02:52:08.583626 16575 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786416707705494-12843-0/minicluster-data/ts-0/wal/instance:
uuid: "dfef187d3b2e432da1643c3bad3ed0e8"
format_stamp: "Formatted at 2026-08-11 02:52:08 on dist-test-slave-9gcw"
server_key: "bf0e4e33ca60cdfea3aa4490886ecf07"
server_key_iv: "2e4d346a52bcca645c2d29107226c17e"
server_key_version: "encryptionkey@0"
I20260811 02:52:08.587769 16575 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.004s
I20260811 02:52:08.590443 16591 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 02:52:08.591784 16575 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260811 02:52:08.591917 16575 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786416707705494-12843-0/minicluster-data/ts-0/data,/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786416707705494-12843-0/minicluster-data/ts-0/wal
uuid: "dfef187d3b2e432da1643c3bad3ed0e8"
format_stamp: "Formatted at 2026-08-11 02:52:08 on dist-test-slave-9gcw"
server_key: "bf0e4e33ca60cdfea3aa4490886ecf07"
server_key_iv: "2e4d346a52bcca645c2d29107226c17e"
server_key_version: "encryptionkey@0"
I20260811 02:52:08.592031 16575 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786416707705494-12843-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786416707705494-12843-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786416707705494-12843-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 02:52:08.609347 16575 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 02:52:08.612586 16575 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 02:52:08.612829 16575 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 02:52:08.613572 16575 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 02:52:08.614495 16575 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 02:52:08.614545 16575 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 02:52:08.614619 16575 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 02:52:08.614655 16575 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 02:52:08.625064 16575 rpc_server.cc:307] RPC server started. Bound to: 127.12.138.193:32861
I20260811 02:52:08.625085 16721 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.138.193:32861 every 8 connection(s)
I20260811 02:52:08.626219 16575 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786416707705494-12843-0/minicluster-data/ts-0/data/info.pb
I20260811 02:52:08.630807 12843 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDawEwS/build/debug/bin/kudu as pid 16575
I20260811 02:52:08.630955 12843 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786416707705494-12843-0/minicluster-data/ts-0/wal/instance
I20260811 02:52:08.631280 12843 external_mini_cluster.cc:1428] Setting key 95246419e04ae7d489806ebaa244e52d
Aug 11 02:52:08 dist-test-slave-9gcw krb5kdc[16453](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416728, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.138.193@KRBTEST.COM for kudu/127.12.138.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.12.138.194@KRBTEST.COM; defaulting to no policy
I20260811 02:52:08.643465 16548 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:52:08.628238 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.138.193:43925 (local address 127.12.138.254:46489)
0811 02:52:08.628413 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0811 02:52:08.628417 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 02:52:08.629672 (+ 1255us) server_negotiation.cc:407] Connection header received
0811 02:52:08.631554 (+ 1882us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:52:08.631557 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:52:08.631609 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:52:08.631677 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:52:08.633224 (+ 1547us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:52:08.633757 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:52:08.635602 (+ 1845us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:52:08.636016 (+ 414us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:52:08.636140 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:52:08.638904 (+ 2764us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:52:08.638911 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:52:08.638917 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:52:08.638956 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:52:08.640756 (+ 1800us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:52:08.641470 (+ 714us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:52:08.641476 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:52:08.641481 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:52:08.641592 (+ 111us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:52:08.642034 (+ 442us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:52:08.642038 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:52:08.642040 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:52:08.642311 (+ 271us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:52:08.642432 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0811 02:52:08.643062 (+ 630us) server_negotiation.cc:299] Negotiation successful
0811 02:52:08.643223 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260811 02:52:08.644335 16724 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:52:08.628615 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.138.254:46489 (local address 127.12.138.193:43925)
0811 02:52:08.629124 (+ 509us) negotiation.cc:107] Waiting for socket to connect
0811 02:52:08.629368 (+ 244us) client_negotiation.cc:174] Beginning negotiation
0811 02:52:08.631260 (+ 1892us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:52:08.631871 (+ 611us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:52:08.631878 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:52:08.632354 (+ 476us) client_negotiation.cc:189] Negotiated authn=SASL
0811 02:52:08.633036 (+ 682us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:52:08.633058 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:52:08.633912 (+ 854us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:52:08.633918 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:52:08.635460 (+ 1542us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:52:08.635471 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:52:08.636175 (+ 704us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:52:08.636177 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:52:08.636339 (+ 162us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:52:08.637019 (+ 680us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 02:52:08.637036 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 02:52:08.638682 (+ 1646us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 02:52:08.640916 (+ 2234us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:52:08.640922 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:52:08.640936 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:52:08.641344 (+ 408us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:52:08.641735 (+ 391us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:52:08.641738 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:52:08.641740 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:52:08.641902 (+ 162us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:52:08.642451 (+ 549us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 02:52:08.642457 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 02:52:08.642757 (+ 300us) client_negotiation.cc:769] Sending connection context
0811 02:52:08.642948 (+ 191us) client_negotiation.cc:240] Negotiation successful
0811 02:52:08.643344 (+ 396us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":309,"thread_start_us":133,"threads_started":1}
I20260811 02:52:08.645888 16722 heartbeater.cc:347] Connected to a master server at 127.12.138.254:46489
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.138.194@KRBTEST.COM" created.
I20260811 02:52:08.646176 16722 heartbeater.cc:464] Registering TS with master...
I20260811 02:52:08.646852 16722 heartbeater.cc:511] Master 127.12.138.254:46489 requested a full tablet report, sending...
I20260811 02:52:08.648675 16505 ts_manager.cc:194] Registered new tserver with Master: dfef187d3b2e432da1643c3bad3ed0e8 (127.12.138.193:32861)
I20260811 02:52:08.649914 16505 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.12.138.193@KRBTEST.COM'} at 127.12.138.193:43925
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.138.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786416707705494-12843-0/krb5kdc/kudu_127.12.138.194.keytab.
Entry for principal kudu/127.12.138.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786416707705494-12843-0/krb5kdc/kudu_127.12.138.194.keytab.
I20260811 02:52:08.664745 12843 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDawEwS/build/debug/bin/kudu
/tmp/dist-test-taskDawEwS/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786416707705494-12843-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786416707705494-12843-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786416707705494-12843-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786416707705494-12843-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.138.194:0
--local_ip_for_outbound_sockets=127.12.138.194
--webserver_interface=127.12.138.194
--webserver_port=0
--tserver_master_addrs=127.12.138.254:46489
--keytab_file=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786416707705494-12843-0/krb5kdc/kudu_127.12.138.194.keytab
--principal=kudu/127.12.138.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.138.212:36249
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786416707705494-12843-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786416707705494-12843-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786416707705494-12843-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 02:52:08.782703 16728 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 02:52:08.783107 16728 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 02:52:08.783203 16728 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 02:52:08.787921 16728 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 02:52:08.788040 16728 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 02:52:08.788154 16728 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.138.194
I20260811 02:52:08.794750 16728 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.138.212:36249
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786416707705494-12843-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786416707705494-12843-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786416707705494-12843-0/krb5kdc/kudu_127.12.138.194.keytab
--principal=kudu/127.12.138.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.138.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786416707705494-12843-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.12.138.194
--webserver_port=0
--tserver_master_addrs=127.12.138.254:46489
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16728
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.138.194
--log_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786416707705494-12843-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 02:09:49 UTC on ccycloud.quasar-fsnqqc.root.comops.site
build id 4579570
I20260811 02:52:08.796156 16728 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 02:52:08.797463 16728 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 02:52:08.808256 16733 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:52:08.808276 16736 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:52:08.808391 16734 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:52:08.808475 16728 server_base.cc:1034] running on GCE node
I20260811 02:52:08.809113 16728 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 02:52:08.809877 16728 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 02:52:08.811084 16728 hybrid_clock.cc:648] HybridClock initialized: now 1786416728811069 us; error 27 us; skew 500 ppm
Aug 11 02:52:08 dist-test-slave-9gcw krb5kdc[16453](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416728, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.138.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 02:52:08.814268 16728 init.cc:377] Logged in from keytab as kudu/127.12.138.194@KRBTEST.COM (short username kudu)
I20260811 02:52:08.815150 16728 webserver.cc:460] Webserver started at http://127.12.138.194:37673/ using document root <none> and password file <none>
I20260811 02:52:08.815785 16728 fs_manager.cc:359] Metadata directory not provided
I20260811 02:52:08.815886 16728 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 02:52:08.816131 16728 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 02:52:08.818228 16728 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786416707705494-12843-0/minicluster-data/ts-1/data/instance:
uuid: "d38f7b7eded94ed58d3b983b18101639"
format_stamp: "Formatted at 2026-08-11 02:52:08 on dist-test-slave-9gcw"
server_key: "58390e976f9bfe298b5d52b073b905d9"
server_key_iv: "e1c5cf614f1848427591917322196c49"
server_key_version: "encryptionkey@0"
I20260811 02:52:08.818813 16728 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786416707705494-12843-0/minicluster-data/ts-1/wal/instance:
uuid: "d38f7b7eded94ed58d3b983b18101639"
format_stamp: "Formatted at 2026-08-11 02:52:08 on dist-test-slave-9gcw"
server_key: "58390e976f9bfe298b5d52b073b905d9"
server_key_iv: "e1c5cf614f1848427591917322196c49"
server_key_version: "encryptionkey@0"
I20260811 02:52:08.822705 16728 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260811 02:52:08.825356 16744 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 02:52:08.826601 16728 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 02:52:08.826738 16728 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786416707705494-12843-0/minicluster-data/ts-1/data,/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786416707705494-12843-0/minicluster-data/ts-1/wal
uuid: "d38f7b7eded94ed58d3b983b18101639"
format_stamp: "Formatted at 2026-08-11 02:52:08 on dist-test-slave-9gcw"
server_key: "58390e976f9bfe298b5d52b073b905d9"
server_key_iv: "e1c5cf614f1848427591917322196c49"
server_key_version: "encryptionkey@0"
I20260811 02:52:08.826857 16728 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786416707705494-12843-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786416707705494-12843-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786416707705494-12843-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 02:52:08.841784 16728 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 02:52:08.845623 16728 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 02:52:08.845889 16728 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 02:52:08.846588 16728 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 02:52:08.847610 16728 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 02:52:08.847684 16728 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 02:52:08.847746 16728 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 02:52:08.847765 16728 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 02:52:08.857482 16728 rpc_server.cc:307] RPC server started. Bound to: 127.12.138.194:34081
I20260811 02:52:08.857528 16876 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.138.194:34081 every 8 connection(s)
I20260811 02:52:08.858506 16728 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786416707705494-12843-0/minicluster-data/ts-1/data/info.pb
I20260811 02:52:08.863263 12843 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDawEwS/build/debug/bin/kudu as pid 16728
I20260811 02:52:08.863397 12843 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786416707705494-12843-0/minicluster-data/ts-1/wal/instance
I20260811 02:52:08.863695 12843 external_mini_cluster.cc:1428] Setting key 721324bd45b1d403a177789a59932ff3
Aug 11 02:52:08 dist-test-slave-9gcw krb5kdc[16453](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416728, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.138.194@KRBTEST.COM for kudu/127.12.138.254@KRBTEST.COM
I20260811 02:52:08.875726 16548 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:52:08.860620 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.138.194:32969 (local address 127.12.138.254:46489)
0811 02:52:08.860903 (+ 283us) server_negotiation.cc:206] Beginning negotiation
0811 02:52:08.860907 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 02:52:08.861615 (+ 708us) server_negotiation.cc:407] Connection header received
0811 02:52:08.863116 (+ 1501us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:52:08.863120 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:52:08.863173 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:52:08.863262 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:52:08.864872 (+ 1610us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:52:08.865679 (+ 807us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:52:08.867527 (+ 1848us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:52:08.868104 (+ 577us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:52:08.868250 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:52:08.871511 (+ 3261us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:52:08.871515 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:52:08.871517 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:52:08.871547 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:52:08.873077 (+ 1530us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:52:08.873849 (+ 772us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:52:08.873852 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:52:08.873853 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:52:08.873932 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:52:08.874405 (+ 473us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:52:08.874408 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:52:08.874409 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:52:08.874601 (+ 192us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:52:08.874678 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0811 02:52:08.875489 (+ 811us) server_negotiation.cc:299] Negotiation successful
0811 02:52:08.875600 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":183}
WARNING: no policy specified for kudu/127.12.138.195@KRBTEST.COM; defaulting to no policy
I20260811 02:52:08.876919 16879 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:52:08.860929 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.138.254:46489 (local address 127.12.138.194:32969)
0811 02:52:08.861451 (+ 522us) negotiation.cc:107] Waiting for socket to connect
0811 02:52:08.861494 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0811 02:52:08.862836 (+ 1342us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:52:08.863493 (+ 657us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:52:08.863503 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:52:08.864040 (+ 537us) client_negotiation.cc:189] Negotiated authn=SASL
0811 02:52:08.864691 (+ 651us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:52:08.864708 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:52:08.865835 (+ 1127us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:52:08.865842 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:52:08.867362 (+ 1520us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:52:08.867377 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:52:08.868271 (+ 894us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:52:08.868274 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:52:08.868433 (+ 159us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:52:08.869094 (+ 661us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 02:52:08.869110 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0811 02:52:08.871314 (+ 2204us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 02:52:08.873254 (+ 1940us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:52:08.873265 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:52:08.873289 (+ 24us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:52:08.873730 (+ 441us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:52:08.874039 (+ 309us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:52:08.874043 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:52:08.874048 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:52:08.874285 (+ 237us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:52:08.874721 (+ 436us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 02:52:08.874731 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 02:52:08.875155 (+ 424us) client_negotiation.cc:769] Sending connection context
0811 02:52:08.875448 (+ 293us) client_negotiation.cc:240] Negotiation successful
0811 02:52:08.875871 (+ 423us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":323,"thread_start_us":139,"threads_started":1}
I20260811 02:52:08.878270 16877 heartbeater.cc:347] Connected to a master server at 127.12.138.254:46489
I20260811 02:52:08.878527 16877 heartbeater.cc:464] Registering TS with master...
I20260811 02:52:08.879218 16877 heartbeater.cc:511] Master 127.12.138.254:46489 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.138.195@KRBTEST.COM" created.
I20260811 02:52:08.880602 16505 ts_manager.cc:194] Registered new tserver with Master: d38f7b7eded94ed58d3b983b18101639 (127.12.138.194:34081)
I20260811 02:52:08.881522 16505 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.12.138.194@KRBTEST.COM'} at 127.12.138.194:32969
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.138.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786416707705494-12843-0/krb5kdc/kudu_127.12.138.195.keytab.
Entry for principal kudu/127.12.138.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786416707705494-12843-0/krb5kdc/kudu_127.12.138.195.keytab.
I20260811 02:52:08.900445 12843 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDawEwS/build/debug/bin/kudu
/tmp/dist-test-taskDawEwS/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786416707705494-12843-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786416707705494-12843-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786416707705494-12843-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786416707705494-12843-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.138.195:0
--local_ip_for_outbound_sockets=127.12.138.195
--webserver_interface=127.12.138.195
--webserver_port=0
--tserver_master_addrs=127.12.138.254:46489
--keytab_file=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786416707705494-12843-0/krb5kdc/kudu_127.12.138.195.keytab
--principal=kudu/127.12.138.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.138.212:36249
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786416707705494-12843-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786416707705494-12843-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786416707705494-12843-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 02:52:09.023406 16883 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 02:52:09.023816 16883 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 02:52:09.023910 16883 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 02:52:09.028458 16883 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 02:52:09.028550 16883 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 02:52:09.028688 16883 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.138.195
I20260811 02:52:09.034461 16883 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.138.212:36249
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786416707705494-12843-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786416707705494-12843-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786416707705494-12843-0/krb5kdc/kudu_127.12.138.195.keytab
--principal=kudu/127.12.138.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.138.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786416707705494-12843-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.12.138.195
--webserver_port=0
--tserver_master_addrs=127.12.138.254:46489
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16883
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.138.195
--log_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786416707705494-12843-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 02:09:49 UTC on ccycloud.quasar-fsnqqc.root.comops.site
build id 4579570
I20260811 02:52:09.035689 16883 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 02:52:09.036821 16883 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 02:52:09.047612 16893 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:52:09.047616 16890 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:52:09.047616 16889 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:52:09.048270 16883 server_base.cc:1034] running on GCE node
I20260811 02:52:09.048739 16883 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 02:52:09.049461 16883 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 02:52:09.050701 16883 hybrid_clock.cc:648] HybridClock initialized: now 1786416729050583 us; error 60 us; skew 500 ppm
Aug 11 02:52:09 dist-test-slave-9gcw krb5kdc[16453](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416729, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.138.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 02:52:09.053799 16883 init.cc:377] Logged in from keytab as kudu/127.12.138.195@KRBTEST.COM (short username kudu)
I20260811 02:52:09.054781 16883 webserver.cc:460] Webserver started at http://127.12.138.195:42053/ using document root <none> and password file <none>
I20260811 02:52:09.055367 16883 fs_manager.cc:359] Metadata directory not provided
I20260811 02:52:09.055433 16883 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 02:52:09.055610 16883 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 02:52:09.057822 16883 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786416707705494-12843-0/minicluster-data/ts-2/data/instance:
uuid: "064345260cc3484085975e8f83876e1f"
format_stamp: "Formatted at 2026-08-11 02:52:09 on dist-test-slave-9gcw"
server_key: "2711fd64ee074a1d1b52d81050d07525"
server_key_iv: "fa577eb02ee2045b56b7c8b373524da6"
server_key_version: "encryptionkey@0"
I20260811 02:52:09.058378 16883 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786416707705494-12843-0/minicluster-data/ts-2/wal/instance:
uuid: "064345260cc3484085975e8f83876e1f"
format_stamp: "Formatted at 2026-08-11 02:52:09 on dist-test-slave-9gcw"
server_key: "2711fd64ee074a1d1b52d81050d07525"
server_key_iv: "fa577eb02ee2045b56b7c8b373524da6"
server_key_version: "encryptionkey@0"
I20260811 02:52:09.062383 16883 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260811 02:52:09.065073 16902 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 02:52:09.066557 16883 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 02:52:09.066708 16883 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786416707705494-12843-0/minicluster-data/ts-2/data,/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786416707705494-12843-0/minicluster-data/ts-2/wal
uuid: "064345260cc3484085975e8f83876e1f"
format_stamp: "Formatted at 2026-08-11 02:52:09 on dist-test-slave-9gcw"
server_key: "2711fd64ee074a1d1b52d81050d07525"
server_key_iv: "fa577eb02ee2045b56b7c8b373524da6"
server_key_version: "encryptionkey@0"
I20260811 02:52:09.066828 16883 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786416707705494-12843-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786416707705494-12843-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786416707705494-12843-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 02:52:09.085928 16883 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 02:52:09.088966 16883 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 02:52:09.089214 16883 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 02:52:09.089840 16883 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 02:52:09.090737 16883 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 02:52:09.090812 16883 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 02:52:09.090888 16883 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 02:52:09.090932 16883 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 02:52:09.100342 16883 rpc_server.cc:307] RPC server started. Bound to: 127.12.138.195:33335
I20260811 02:52:09.100371 17038 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.138.195:33335 every 8 connection(s)
I20260811 02:52:09.101277 16883 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786416707705494-12843-0/minicluster-data/ts-2/data/info.pb
I20260811 02:52:09.108225 12843 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDawEwS/build/debug/bin/kudu as pid 16883
I20260811 02:52:09.108400 12843 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786416707705494-12843-0/minicluster-data/ts-2/wal/instance
I20260811 02:52:09.108723 12843 external_mini_cluster.cc:1428] Setting key 0d3bd74ec42d60373178f23a7afa5f0f
Aug 11 02:52:09 dist-test-slave-9gcw krb5kdc[16453](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416729, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.138.195@KRBTEST.COM for kudu/127.12.138.254@KRBTEST.COM
I20260811 02:52:09.116532 16548 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:52:09.103094 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.138.195:35719 (local address 127.12.138.254:46489)
0811 02:52:09.103240 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0811 02:52:09.103244 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 02:52:09.104163 (+ 919us) server_negotiation.cc:407] Connection header received
0811 02:52:09.105697 (+ 1534us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:52:09.105701 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:52:09.105761 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:52:09.105871 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:52:09.107077 (+ 1206us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:52:09.107635 (+ 558us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:52:09.108932 (+ 1297us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:52:09.109263 (+ 331us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:52:09.109401 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:52:09.112522 (+ 3121us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:52:09.112526 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:52:09.112528 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:52:09.112554 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:52:09.114422 (+ 1868us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:52:09.114976 (+ 554us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:52:09.114980 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:52:09.114981 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:52:09.115066 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:52:09.115431 (+ 365us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:52:09.115434 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:52:09.115435 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:52:09.115627 (+ 192us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:52:09.115712 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0811 02:52:09.116288 (+ 576us) server_negotiation.cc:299] Negotiation successful
0811 02:52:09.116406 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260811 02:52:09.117471 17041 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:52:09.103485 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.138.254:46489 (local address 127.12.138.195:35719)
0811 02:52:09.103978 (+ 493us) negotiation.cc:107] Waiting for socket to connect
0811 02:52:09.104023 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0811 02:52:09.105476 (+ 1453us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:52:09.106053 (+ 577us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:52:09.106060 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:52:09.106437 (+ 377us) client_negotiation.cc:189] Negotiated authn=SASL
0811 02:52:09.106933 (+ 496us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:52:09.106946 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:52:09.107764 (+ 818us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:52:09.107767 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:52:09.108779 (+ 1012us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:52:09.108789 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:52:09.109517 (+ 728us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:52:09.109521 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:52:09.109761 (+ 240us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:52:09.110519 (+ 758us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 02:52:09.110543 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0811 02:52:09.112330 (+ 1787us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 02:52:09.114541 (+ 2211us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:52:09.114546 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:52:09.114560 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:52:09.114841 (+ 281us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:52:09.115164 (+ 323us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:52:09.115166 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:52:09.115167 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:52:09.115309 (+ 142us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:52:09.115753 (+ 444us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 02:52:09.115759 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 02:52:09.116007 (+ 248us) client_negotiation.cc:769] Sending connection context
0811 02:52:09.116181 (+ 174us) client_negotiation.cc:240] Negotiation successful
0811 02:52:09.116457 (+ 276us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":290,"thread_start_us":116,"threads_started":1}
I20260811 02:52:09.118884 17039 heartbeater.cc:347] Connected to a master server at 127.12.138.254:46489
I20260811 02:52:09.119167 17039 heartbeater.cc:464] Registering TS with master...
I20260811 02:52:09.119701 17039 heartbeater.cc:511] Master 127.12.138.254:46489 requested a full tablet report, sending...
I20260811 02:52:09.120855 16505 ts_manager.cc:194] Registered new tserver with Master: 064345260cc3484085975e8f83876e1f (127.12.138.195:33335)
I20260811 02:52:09.121526 16505 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.12.138.195@KRBTEST.COM'} at 127.12.138.195:35719
I20260811 02:52:09.123263 12843 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 11 02:52:09 dist-test-slave-9gcw krb5kdc[16453](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416729, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for joe-interloper@KRBTEST.COM:
Aug 11 02:52:09 dist-test-slave-9gcw krb5kdc[16453](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416729, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.12.138.254@KRBTEST.COM
I20260811 02:52:09.153766 16548 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:52:09.144295 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58368 (local address 127.12.138.254:46489)
0811 02:52:09.144475 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0811 02:52:09.144479 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 02:52:09.144549 (+ 70us) server_negotiation.cc:407] Connection header received
0811 02:52:09.144803 (+ 254us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:52:09.144815 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:52:09.144866 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:52:09.144969 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:52:09.145948 (+ 979us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:52:09.146493 (+ 545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:52:09.147417 (+ 924us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:52:09.147662 (+ 245us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:52:09.147788 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:52:09.150687 (+ 2899us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:52:09.150690 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:52:09.150692 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:52:09.150720 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:52:09.152056 (+ 1336us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:52:09.152458 (+ 402us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:52:09.152461 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:52:09.152462 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:52:09.152547 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:52:09.152826 (+ 279us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:52:09.152830 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:52:09.152831 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:52:09.153075 (+ 244us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:52:09.153289 (+ 214us) server_negotiation.cc:1035] Waiting for connection context
0811 02:52:09.153498 (+ 209us) server_negotiation.cc:299] Negotiation successful
0811 02:52:09.153616 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":86}
W20260811 02:52:09.154219 16505 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.ConnectToMaster from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:58368
I20260811 02:52:09.155388 12843 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDawEwS/build/debug/bin/kudu with pid 16575
I20260811 02:52:09.167615 12843 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDawEwS/build/debug/bin/kudu with pid 16728
I20260811 02:52:09.175433 12843 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDawEwS/build/debug/bin/kudu with pid 16883
I20260811 02:52:09.184350 12843 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDawEwS/build/debug/bin/kudu with pid 16467
2026-08-11T02:52:09Z chronyd exiting
[ OK ] SecurityITest.TestUnauthorizedClientKerberosCredentials (2384 ms)
[ RUN ] SecurityITest.TestAuthorizedSuperuser
Loading random data
Initializing database '/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786416707705494-12843-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 02:52:09 dist-test-slave-9gcw krb5kdc[17055](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 02:52:09 dist-test-slave-9gcw krb5kdc[17055](info): set up 2 sockets
Aug 11 02:52:09 dist-test-slave-9gcw krb5kdc[17055](info): commencing operation
krb5kdc: starting...
W20260811 02:52:10.333567 12843 mini_kdc.cc:121] Time spent starting KDC: real 1.124s user 0.007s sys 0.000s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786416707705494-12843-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786416707705494-12843-0/krb5kdc/test-user.keytab.
Aug 11 02:52:10 dist-test-slave-9gcw krb5kdc[17055](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416730, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T02:52:10Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T02:52:10Z Disabled control of system clock
WARNING: no policy specified for kudu/127.12.138.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.138.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.138.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786416707705494-12843-0/krb5kdc/kudu_127.12.138.254.keytab.
Entry for principal kudu/127.12.138.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786416707705494-12843-0/krb5kdc/kudu_127.12.138.254.keytab.
I20260811 02:52:10.472385 12843 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDawEwS/build/debug/bin/kudu
/tmp/dist-test-taskDawEwS/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786416707705494-12843-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786416707705494-12843-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786416707705494-12843-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786416707705494-12843-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.138.254:39439
--webserver_interface=127.12.138.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786416707705494-12843-0/krb5kdc/kudu_127.12.138.254.keytab
--principal=kudu/127.12.138.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.138.212:46727
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.12.138.254:39439
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786416707705494-12843-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786416707705494-12843-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786416707705494-12843-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 02:52:10.591962 17068 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 02:52:10.592295 17068 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 02:52:10.592389 17068 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 02:52:10.596580 17068 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 02:52:10.596701 17068 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 02:52:10.596748 17068 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 02:52:10.596817 17068 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 02:52:10.596861 17068 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 02:52:10.602376 17068 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.138.212:46727
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786416707705494-12843-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786416707705494-12843-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.12.138.254:39439
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786416707705494-12843-0/krb5kdc/kudu_127.12.138.254.keytab
--principal=kudu/127.12.138.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.138.254:39439
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786416707705494-12843-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.12.138.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17068
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786416707705494-12843-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 02:09:49 UTC on ccycloud.quasar-fsnqqc.root.comops.site
build id 4579570
I20260811 02:52:10.603538 17068 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 02:52:10.604624 17068 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 02:52:10.613641 17077 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:52:10.613699 17079 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:52:10.613914 17075 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:52:10.614063 17068 server_base.cc:1034] running on GCE node
I20260811 02:52:10.614704 17068 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 02:52:10.616014 17068 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 02:52:10.617244 17068 hybrid_clock.cc:648] HybridClock initialized: now 1786416730617222 us; error 39 us; skew 500 ppm
Aug 11 02:52:10 dist-test-slave-9gcw krb5kdc[17055](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416730, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.138.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 02:52:10.620342 17068 init.cc:377] Logged in from keytab as kudu/127.12.138.254@KRBTEST.COM (short username kudu)
I20260811 02:52:10.621553 17068 webserver.cc:460] Webserver started at http://127.12.138.254:35625/ using document root <none> and password file <none>
I20260811 02:52:10.622221 17068 fs_manager.cc:359] Metadata directory not provided
I20260811 02:52:10.622327 17068 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 02:52:10.622603 17068 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 02:52:10.625020 17068 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786416707705494-12843-0/minicluster-data/master-0/data/instance:
uuid: "20c75e3f66d348ae94807efe7dca6cf4"
format_stamp: "Formatted at 2026-08-11 02:52:10 on dist-test-slave-9gcw"
server_key: "8ae707fb203bee3ac0fd21cf092a8787"
server_key_iv: "d45f28c034265c1ee2153fd1531a39d8"
server_key_version: "encryptionkey@0"
I20260811 02:52:10.625847 17068 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786416707705494-12843-0/minicluster-data/master-0/wal/instance:
uuid: "20c75e3f66d348ae94807efe7dca6cf4"
format_stamp: "Formatted at 2026-08-11 02:52:10 on dist-test-slave-9gcw"
server_key: "8ae707fb203bee3ac0fd21cf092a8787"
server_key_iv: "d45f28c034265c1ee2153fd1531a39d8"
server_key_version: "encryptionkey@0"
I20260811 02:52:10.630584 17068 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260811 02:52:10.633740 17086 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 02:52:10.635118 17068 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260811 02:52:10.635318 17068 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786416707705494-12843-0/minicluster-data/master-0/data,/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786416707705494-12843-0/minicluster-data/master-0/wal
uuid: "20c75e3f66d348ae94807efe7dca6cf4"
format_stamp: "Formatted at 2026-08-11 02:52:10 on dist-test-slave-9gcw"
server_key: "8ae707fb203bee3ac0fd21cf092a8787"
server_key_iv: "d45f28c034265c1ee2153fd1531a39d8"
server_key_version: "encryptionkey@0"
I20260811 02:52:10.635473 17068 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786416707705494-12843-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786416707705494-12843-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786416707705494-12843-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 02:52:10.645888 17068 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 02:52:10.649356 17068 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 02:52:10.649619 17068 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 02:52:10.658124 17068 rpc_server.cc:307] RPC server started. Bound to: 127.12.138.254:39439
I20260811 02:52:10.658154 17151 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.138.254:39439 every 8 connection(s)
I20260811 02:52:10.659268 17068 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786416707705494-12843-0/minicluster-data/master-0/data/info.pb
I20260811 02:52:10.660269 12843 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDawEwS/build/debug/bin/kudu as pid 17068
I20260811 02:52:10.660406 12843 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786416707705494-12843-0/minicluster-data/master-0/wal/instance
I20260811 02:52:10.660733 12843 external_mini_cluster.cc:1428] Setting key a0cd2dd10a11c410ead70be52300adad
I20260811 02:52:10.662994 17152 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 02:52:10.668867 17152 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 20c75e3f66d348ae94807efe7dca6cf4: Bootstrap starting.
Aug 11 02:52:10 dist-test-slave-9gcw krb5kdc[17055](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416730, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.12.138.254@KRBTEST.COM
I20260811 02:52:10.671696 17152 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 20c75e3f66d348ae94807efe7dca6cf4: Neither blocks nor log segments found. Creating new log.
I20260811 02:52:10.672467 17152 log.cc:824] T 00000000000000000000000000000000 P 20c75e3f66d348ae94807efe7dca6cf4: Log is configured to *not* fsync() on all Append() calls
I20260811 02:52:10.675046 17152 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 20c75e3f66d348ae94807efe7dca6cf4: No bootstrap required, opened a new log
I20260811 02:52:10.677703 17156 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:52:10.662343 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53042 (local address 127.12.138.254:39439)
0811 02:52:10.662918 (+ 575us) server_negotiation.cc:206] Beginning negotiation
0811 02:52:10.662936 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0811 02:52:10.662975 (+ 39us) server_negotiation.cc:407] Connection header received
0811 02:52:10.663758 (+ 783us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:52:10.663776 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:52:10.664136 (+ 360us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:52:10.664522 (+ 386us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:52:10.665517 (+ 995us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:52:10.666644 (+ 1127us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:52:10.667642 (+ 998us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:52:10.668212 (+ 570us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:52:10.668430 (+ 218us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:52:10.671638 (+ 3208us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:52:10.671644 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:52:10.671662 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:52:10.671706 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:52:10.674086 (+ 2380us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:52:10.674678 (+ 592us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:52:10.674683 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:52:10.674688 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:52:10.674818 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:52:10.675195 (+ 377us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:52:10.675199 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:52:10.675201 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:52:10.675755 (+ 554us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:52:10.675919 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0811 02:52:10.676270 (+ 351us) server_negotiation.cc:299] Negotiation successful
0811 02:52:10.676694 (+ 424us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":421,"thread_start_us":155,"threads_started":1}
I20260811 02:52:10.679060 17152 raft_consensus.cc:348] T 00000000000000000000000000000000 P 20c75e3f66d348ae94807efe7dca6cf4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "20c75e3f66d348ae94807efe7dca6cf4" member_type: VOTER last_known_addr { host: "127.12.138.254" port: 39439 } }
I20260811 02:52:10.679282 17152 raft_consensus.cc:374] T 00000000000000000000000000000000 P 20c75e3f66d348ae94807efe7dca6cf4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 02:52:10.679347 17152 raft_consensus.cc:729] T 00000000000000000000000000000000 P 20c75e3f66d348ae94807efe7dca6cf4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 20c75e3f66d348ae94807efe7dca6cf4, State: Initialized, Role: FOLLOWER
I20260811 02:52:10.679762 17152 consensus_queue.cc:260] T 00000000000000000000000000000000 P 20c75e3f66d348ae94807efe7dca6cf4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "20c75e3f66d348ae94807efe7dca6cf4" member_type: VOTER last_known_addr { host: "127.12.138.254" port: 39439 } }
I20260811 02:52:10.679895 17152 raft_consensus.cc:388] T 00000000000000000000000000000000 P 20c75e3f66d348ae94807efe7dca6cf4 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 02:52:10.679983 17152 raft_consensus.cc:482] T 00000000000000000000000000000000 P 20c75e3f66d348ae94807efe7dca6cf4 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 02:52:10.680111 17152 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 20c75e3f66d348ae94807efe7dca6cf4 [term 0 FOLLOWER]: Advancing to term 1
I20260811 02:52:10.681046 17152 raft_consensus.cc:504] T 00000000000000000000000000000000 P 20c75e3f66d348ae94807efe7dca6cf4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "20c75e3f66d348ae94807efe7dca6cf4" member_type: VOTER last_known_addr { host: "127.12.138.254" port: 39439 } }
I20260811 02:52:10.681413 17152 leader_election.cc:302] T 00000000000000000000000000000000 P 20c75e3f66d348ae94807efe7dca6cf4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 20c75e3f66d348ae94807efe7dca6cf4; no voters:
I20260811 02:52:10.681732 17152 leader_election.cc:288] T 00000000000000000000000000000000 P 20c75e3f66d348ae94807efe7dca6cf4 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 02:52:10.681936 17158 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 20c75e3f66d348ae94807efe7dca6cf4 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 02:52:10.682345 17158 raft_consensus.cc:686] T 00000000000000000000000000000000 P 20c75e3f66d348ae94807efe7dca6cf4 [term 1 LEADER]: Becoming Leader. State: Replica: 20c75e3f66d348ae94807efe7dca6cf4, State: Running, Role: LEADER
I20260811 02:52:10.682675 17158 consensus_queue.cc:237] T 00000000000000000000000000000000 P 20c75e3f66d348ae94807efe7dca6cf4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "20c75e3f66d348ae94807efe7dca6cf4" member_type: VOTER last_known_addr { host: "127.12.138.254" port: 39439 } }
I20260811 02:52:10.683077 17152 sys_catalog.cc:564] T 00000000000000000000000000000000 P 20c75e3f66d348ae94807efe7dca6cf4 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 02:52:10.685276 17160 sys_catalog.cc:455] T 00000000000000000000000000000000 P 20c75e3f66d348ae94807efe7dca6cf4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 20c75e3f66d348ae94807efe7dca6cf4. Latest consensus state: current_term: 1 leader_uuid: "20c75e3f66d348ae94807efe7dca6cf4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "20c75e3f66d348ae94807efe7dca6cf4" member_type: VOTER last_known_addr { host: "127.12.138.254" port: 39439 } } }
I20260811 02:52:10.685428 17160 sys_catalog.cc:458] T 00000000000000000000000000000000 P 20c75e3f66d348ae94807efe7dca6cf4 [sys.catalog]: This master's current role is: LEADER
I20260811 02:52:10.685674 17159 sys_catalog.cc:455] T 00000000000000000000000000000000 P 20c75e3f66d348ae94807efe7dca6cf4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "20c75e3f66d348ae94807efe7dca6cf4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "20c75e3f66d348ae94807efe7dca6cf4" member_type: VOTER last_known_addr { host: "127.12.138.254" port: 39439 } } }
I20260811 02:52:10.686007 17159 sys_catalog.cc:458] T 00000000000000000000000000000000 P 20c75e3f66d348ae94807efe7dca6cf4 [sys.catalog]: This master's current role is: LEADER
W20260811 02:52:10.688767 17173 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 20c75e3f66d348ae94807efe7dca6cf4: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 02:52:10.688854 17173 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 02:52:10.688943 17175 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 02:52:10.689839 17175 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 02:52:10.695842 17175 catalog_manager.cc:1347] Generated new cluster ID: 0e94a3546b3641caadf8c89f8763f770
I20260811 02:52:10.695932 17175 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 02:52:10.709872 17175 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 02:52:10.710974 17175 catalog_manager.cc:1504] Loading token signing keys...
I20260811 02:52:10.721316 17175 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 20c75e3f66d348ae94807efe7dca6cf4: Generated new TSK 0
I20260811 02:52:10.722146 17175 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.12.138.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.138.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.138.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786416707705494-12843-0/krb5kdc/kudu_127.12.138.193.keytab.
Entry for principal kudu/127.12.138.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786416707705494-12843-0/krb5kdc/kudu_127.12.138.193.keytab.
I20260811 02:52:10.761194 12843 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDawEwS/build/debug/bin/kudu
/tmp/dist-test-taskDawEwS/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786416707705494-12843-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786416707705494-12843-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786416707705494-12843-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786416707705494-12843-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.138.193:0
--local_ip_for_outbound_sockets=127.12.138.193
--webserver_interface=127.12.138.193
--webserver_port=0
--tserver_master_addrs=127.12.138.254:39439
--keytab_file=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786416707705494-12843-0/krb5kdc/kudu_127.12.138.193.keytab
--principal=kudu/127.12.138.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.138.212:46727
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786416707705494-12843-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786416707705494-12843-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786416707705494-12843-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 02:52:10.879352 17181 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 02:52:10.879658 17181 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 02:52:10.879704 17181 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 02:52:10.884152 17181 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 02:52:10.884223 17181 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 02:52:10.884342 17181 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.138.193
I20260811 02:52:10.890165 17181 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.138.212:46727
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786416707705494-12843-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786416707705494-12843-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786416707705494-12843-0/krb5kdc/kudu_127.12.138.193.keytab
--principal=kudu/127.12.138.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.138.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786416707705494-12843-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.12.138.193
--webserver_port=0
--tserver_master_addrs=127.12.138.254:39439
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17181
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.138.193
--log_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786416707705494-12843-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 02:09:49 UTC on ccycloud.quasar-fsnqqc.root.comops.site
build id 4579570
I20260811 02:52:10.891340 17181 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 02:52:10.892416 17181 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 02:52:10.902442 17189 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:52:10.902438 17186 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:52:10.902604 17187 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:52:10.902715 17181 server_base.cc:1034] running on GCE node
I20260811 02:52:10.903285 17181 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 02:52:10.903877 17181 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 02:52:10.905040 17181 hybrid_clock.cc:648] HybridClock initialized: now 1786416730905032 us; error 50 us; skew 500 ppm
Aug 11 02:52:10 dist-test-slave-9gcw krb5kdc[17055](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416730, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.138.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 02:52:10.908036 17181 init.cc:377] Logged in from keytab as kudu/127.12.138.193@KRBTEST.COM (short username kudu)
I20260811 02:52:10.909004 17181 webserver.cc:460] Webserver started at http://127.12.138.193:41467/ using document root <none> and password file <none>
I20260811 02:52:10.909616 17181 fs_manager.cc:359] Metadata directory not provided
I20260811 02:52:10.909680 17181 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 02:52:10.909925 17181 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 02:52:10.912063 17181 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786416707705494-12843-0/minicluster-data/ts-0/data/instance:
uuid: "b885943b23b54c8ab02b2163d4b20b36"
format_stamp: "Formatted at 2026-08-11 02:52:10 on dist-test-slave-9gcw"
server_key: "e1ca2c8c7ad7e36a42eb9faf63aaec21"
server_key_iv: "89bd1966d077b9b8c0ac4e5ed495fb7a"
server_key_version: "encryptionkey@0"
I20260811 02:52:10.912676 17181 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786416707705494-12843-0/minicluster-data/ts-0/wal/instance:
uuid: "b885943b23b54c8ab02b2163d4b20b36"
format_stamp: "Formatted at 2026-08-11 02:52:10 on dist-test-slave-9gcw"
server_key: "e1ca2c8c7ad7e36a42eb9faf63aaec21"
server_key_iv: "89bd1966d077b9b8c0ac4e5ed495fb7a"
server_key_version: "encryptionkey@0"
I20260811 02:52:10.916643 17181 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260811 02:52:10.919189 17197 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 02:52:10.920428 17181 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260811 02:52:10.920583 17181 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786416707705494-12843-0/minicluster-data/ts-0/data,/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786416707705494-12843-0/minicluster-data/ts-0/wal
uuid: "b885943b23b54c8ab02b2163d4b20b36"
format_stamp: "Formatted at 2026-08-11 02:52:10 on dist-test-slave-9gcw"
server_key: "e1ca2c8c7ad7e36a42eb9faf63aaec21"
server_key_iv: "89bd1966d077b9b8c0ac4e5ed495fb7a"
server_key_version: "encryptionkey@0"
I20260811 02:52:10.920703 17181 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786416707705494-12843-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786416707705494-12843-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786416707705494-12843-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 02:52:10.937252 17181 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 02:52:10.940586 17181 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 02:52:10.940807 17181 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 02:52:10.941479 17181 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 02:52:10.942348 17181 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 02:52:10.942397 17181 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 02:52:10.942468 17181 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 02:52:10.942499 17181 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 02:52:10.951195 17181 rpc_server.cc:307] RPC server started. Bound to: 127.12.138.193:38145
I20260811 02:52:10.951282 17331 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.138.193:38145 every 8 connection(s)
I20260811 02:52:10.952133 17181 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786416707705494-12843-0/minicluster-data/ts-0/data/info.pb
I20260811 02:52:10.959270 12843 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDawEwS/build/debug/bin/kudu as pid 17181
I20260811 02:52:10.959391 12843 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786416707705494-12843-0/minicluster-data/ts-0/wal/instance
I20260811 02:52:10.959710 12843 external_mini_cluster.cc:1428] Setting key cbe006a650fdc94068c1b5854980c60b
Aug 11 02:52:10 dist-test-slave-9gcw krb5kdc[17055](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416730, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.138.193@KRBTEST.COM for kudu/127.12.138.254@KRBTEST.COM
I20260811 02:52:10.967042 17156 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:52:10.954021 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.138.193:45005 (local address 127.12.138.254:39439)
0811 02:52:10.954157 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0811 02:52:10.954161 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 02:52:10.954977 (+ 816us) server_negotiation.cc:407] Connection header received
0811 02:52:10.956386 (+ 1409us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:52:10.956389 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:52:10.956448 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:52:10.956556 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:52:10.957831 (+ 1275us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:52:10.958388 (+ 557us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:52:10.959641 (+ 1253us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:52:10.959961 (+ 320us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:52:10.960079 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:52:10.962769 (+ 2690us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:52:10.962779 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:52:10.962781 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:52:10.962804 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:52:10.964605 (+ 1801us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:52:10.965291 (+ 686us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:52:10.965293 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:52:10.965295 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:52:10.965379 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:52:10.965785 (+ 406us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:52:10.965787 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:52:10.965788 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:52:10.965970 (+ 182us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:52:10.966047 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0811 02:52:10.966737 (+ 690us) server_negotiation.cc:299] Negotiation successful
0811 02:52:10.966907 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260811 02:52:10.968145 17335 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:52:10.954389 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.138.254:39439 (local address 127.12.138.193:45005)
0811 02:52:10.954835 (+ 446us) negotiation.cc:107] Waiting for socket to connect
0811 02:52:10.954877 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0811 02:52:10.956172 (+ 1295us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:52:10.956730 (+ 558us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:52:10.956737 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:52:10.957096 (+ 359us) client_negotiation.cc:189] Negotiated authn=SASL
0811 02:52:10.957675 (+ 579us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:52:10.957690 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:52:10.958524 (+ 834us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:52:10.958533 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:52:10.959512 (+ 979us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:52:10.959522 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:52:10.960119 (+ 597us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:52:10.960122 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:52:10.960352 (+ 230us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:52:10.961084 (+ 732us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 02:52:10.961105 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0811 02:52:10.962604 (+ 1499us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 02:52:10.964738 (+ 2134us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:52:10.964743 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:52:10.964754 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:52:10.965194 (+ 440us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:52:10.965476 (+ 282us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:52:10.965482 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:52:10.965487 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:52:10.965685 (+ 198us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:52:10.966078 (+ 393us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 02:52:10.966087 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 02:52:10.966442 (+ 355us) client_negotiation.cc:769] Sending connection context
0811 02:52:10.966688 (+ 246us) client_negotiation.cc:240] Negotiation successful
0811 02:52:10.967097 (+ 409us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":262,"thread_start_us":116,"threads_started":1}
I20260811 02:52:10.969450 17332 heartbeater.cc:347] Connected to a master server at 127.12.138.254:39439
I20260811 02:52:10.969686 17332 heartbeater.cc:464] Registering TS with master...
I20260811 02:52:10.970192 17332 heartbeater.cc:511] Master 127.12.138.254:39439 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.12.138.194@KRBTEST.COM; defaulting to no policy
I20260811 02:52:10.971701 17109 ts_manager.cc:194] Registered new tserver with Master: b885943b23b54c8ab02b2163d4b20b36 (127.12.138.193:38145)
I20260811 02:52:10.972839 17109 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.12.138.193@KRBTEST.COM'} at 127.12.138.193:45005
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.138.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.138.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786416707705494-12843-0/krb5kdc/kudu_127.12.138.194.keytab.
Entry for principal kudu/127.12.138.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786416707705494-12843-0/krb5kdc/kudu_127.12.138.194.keytab.
I20260811 02:52:10.990120 12843 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDawEwS/build/debug/bin/kudu
/tmp/dist-test-taskDawEwS/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786416707705494-12843-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786416707705494-12843-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786416707705494-12843-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786416707705494-12843-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.138.194:0
--local_ip_for_outbound_sockets=127.12.138.194
--webserver_interface=127.12.138.194
--webserver_port=0
--tserver_master_addrs=127.12.138.254:39439
--keytab_file=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786416707705494-12843-0/krb5kdc/kudu_127.12.138.194.keytab
--principal=kudu/127.12.138.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.138.212:46727
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786416707705494-12843-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786416707705494-12843-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786416707705494-12843-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 02:52:11.105578 17338 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 02:52:11.105865 17338 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 02:52:11.105911 17338 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 02:52:11.110037 17338 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 02:52:11.110107 17338 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 02:52:11.110167 17338 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.138.194
I20260811 02:52:11.115458 17338 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.138.212:46727
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786416707705494-12843-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786416707705494-12843-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786416707705494-12843-0/krb5kdc/kudu_127.12.138.194.keytab
--principal=kudu/127.12.138.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.138.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786416707705494-12843-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.12.138.194
--webserver_port=0
--tserver_master_addrs=127.12.138.254:39439
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17338
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.138.194
--log_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786416707705494-12843-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 02:09:49 UTC on ccycloud.quasar-fsnqqc.root.comops.site
build id 4579570
I20260811 02:52:11.116664 17338 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 02:52:11.117781 17338 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 02:52:11.127874 17346 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:52:11.128077 17338 server_base.cc:1034] running on GCE node
W20260811 02:52:11.127890 17349 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:52:11.127874 17345 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:52:11.128633 17338 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 02:52:11.129283 17338 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 02:52:11.130486 17338 hybrid_clock.cc:648] HybridClock initialized: now 1786416731130470 us; error 27 us; skew 500 ppm
Aug 11 02:52:11 dist-test-slave-9gcw krb5kdc[17055](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416731, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.138.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 02:52:11.133428 17338 init.cc:377] Logged in from keytab as kudu/127.12.138.194@KRBTEST.COM (short username kudu)
I20260811 02:52:11.134378 17338 webserver.cc:460] Webserver started at http://127.12.138.194:39205/ using document root <none> and password file <none>
I20260811 02:52:11.134956 17338 fs_manager.cc:359] Metadata directory not provided
I20260811 02:52:11.135047 17338 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 02:52:11.135293 17338 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 02:52:11.137545 17338 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786416707705494-12843-0/minicluster-data/ts-1/data/instance:
uuid: "1c9ec454393b4ba484b18cc303b7ca2d"
format_stamp: "Formatted at 2026-08-11 02:52:11 on dist-test-slave-9gcw"
server_key: "88c26ebe026587185af9af5c3c8b2e46"
server_key_iv: "0321c14dd44fedde65fb81506121bbf0"
server_key_version: "encryptionkey@0"
I20260811 02:52:11.138131 17338 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786416707705494-12843-0/minicluster-data/ts-1/wal/instance:
uuid: "1c9ec454393b4ba484b18cc303b7ca2d"
format_stamp: "Formatted at 2026-08-11 02:52:11 on dist-test-slave-9gcw"
server_key: "88c26ebe026587185af9af5c3c8b2e46"
server_key_iv: "0321c14dd44fedde65fb81506121bbf0"
server_key_version: "encryptionkey@0"
I20260811 02:52:11.141974 17338 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260811 02:52:11.144390 17359 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 02:52:11.145704 17338 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260811 02:52:11.145839 17338 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786416707705494-12843-0/minicluster-data/ts-1/data,/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786416707705494-12843-0/minicluster-data/ts-1/wal
uuid: "1c9ec454393b4ba484b18cc303b7ca2d"
format_stamp: "Formatted at 2026-08-11 02:52:11 on dist-test-slave-9gcw"
server_key: "88c26ebe026587185af9af5c3c8b2e46"
server_key_iv: "0321c14dd44fedde65fb81506121bbf0"
server_key_version: "encryptionkey@0"
I20260811 02:52:11.145957 17338 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786416707705494-12843-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786416707705494-12843-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786416707705494-12843-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 02:52:11.170176 17338 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 02:52:11.173424 17338 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 02:52:11.173667 17338 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 02:52:11.174285 17338 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 02:52:11.175100 17338 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 02:52:11.175150 17338 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 02:52:11.175189 17338 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 02:52:11.175204 17338 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 02:52:11.184172 17338 rpc_server.cc:307] RPC server started. Bound to: 127.12.138.194:35121
I20260811 02:52:11.184188 17496 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.138.194:35121 every 8 connection(s)
I20260811 02:52:11.185320 17338 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786416707705494-12843-0/minicluster-data/ts-1/data/info.pb
I20260811 02:52:11.187561 12843 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDawEwS/build/debug/bin/kudu as pid 17338
I20260811 02:52:11.187673 12843 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786416707705494-12843-0/minicluster-data/ts-1/wal/instance
I20260811 02:52:11.188031 12843 external_mini_cluster.cc:1428] Setting key a2e84494284fad3270d3857616a1046c
Aug 11 02:52:11 dist-test-slave-9gcw krb5kdc[17055](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416731, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.138.194@KRBTEST.COM for kudu/127.12.138.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.12.138.195@KRBTEST.COM; defaulting to no policy
I20260811 02:52:11.201654 17156 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:52:11.187182 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.138.194:49441 (local address 127.12.138.254:39439)
0811 02:52:11.187357 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0811 02:52:11.187363 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 02:52:11.188336 (+ 973us) server_negotiation.cc:407] Connection header received
0811 02:52:11.190132 (+ 1796us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:52:11.190134 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:52:11.190178 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:52:11.190238 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:52:11.191888 (+ 1650us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:52:11.192753 (+ 865us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:52:11.194556 (+ 1803us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:52:11.194782 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:52:11.194904 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:52:11.197850 (+ 2946us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:52:11.197853 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:52:11.197856 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:52:11.197888 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:52:11.199400 (+ 1512us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:52:11.200033 (+ 633us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:52:11.200036 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:52:11.200037 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:52:11.200115 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:52:11.200463 (+ 348us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:52:11.200469 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:52:11.200474 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:52:11.200692 (+ 218us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:52:11.200772 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0811 02:52:11.201312 (+ 540us) server_negotiation.cc:299] Negotiation successful
0811 02:52:11.201419 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.138.195@KRBTEST.COM" created.
I20260811 02:52:11.202519 17499 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:52:11.187580 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.138.254:39439 (local address 127.12.138.194:49441)
0811 02:52:11.188154 (+ 574us) negotiation.cc:107] Waiting for socket to connect
0811 02:52:11.188212 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0811 02:52:11.189891 (+ 1679us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:52:11.190429 (+ 538us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:52:11.190440 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:52:11.190997 (+ 557us) client_negotiation.cc:189] Negotiated authn=SASL
0811 02:52:11.191705 (+ 708us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:52:11.191728 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:52:11.192898 (+ 1170us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:52:11.192901 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:52:11.194422 (+ 1521us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:52:11.194437 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:52:11.194958 (+ 521us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:52:11.194967 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:52:11.195209 (+ 242us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:52:11.195874 (+ 665us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 02:52:11.195901 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0811 02:52:11.197681 (+ 1780us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 02:52:11.199554 (+ 1873us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:52:11.199561 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:52:11.199577 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:52:11.199932 (+ 355us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:52:11.200198 (+ 266us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:52:11.200200 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:52:11.200202 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:52:11.200332 (+ 130us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:52:11.200786 (+ 454us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 02:52:11.200808 (+ 22us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 02:52:11.201044 (+ 236us) client_negotiation.cc:769] Sending connection context
0811 02:52:11.201254 (+ 210us) client_negotiation.cc:240] Negotiation successful
0811 02:52:11.201523 (+ 269us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":343,"thread_start_us":123,"threads_started":1}
I20260811 02:52:11.205418 17497 heartbeater.cc:347] Connected to a master server at 127.12.138.254:39439
I20260811 02:52:11.205662 17497 heartbeater.cc:464] Registering TS with master...
I20260811 02:52:11.206139 17497 heartbeater.cc:511] Master 127.12.138.254:39439 requested a full tablet report, sending...
I20260811 02:52:11.207185 17109 ts_manager.cc:194] Registered new tserver with Master: 1c9ec454393b4ba484b18cc303b7ca2d (127.12.138.194:35121)
I20260811 02:52:11.208101 17109 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.12.138.194@KRBTEST.COM'} at 127.12.138.194:49441
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.138.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786416707705494-12843-0/krb5kdc/kudu_127.12.138.195.keytab.
Entry for principal kudu/127.12.138.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786416707705494-12843-0/krb5kdc/kudu_127.12.138.195.keytab.
I20260811 02:52:11.221944 12843 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDawEwS/build/debug/bin/kudu
/tmp/dist-test-taskDawEwS/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786416707705494-12843-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786416707705494-12843-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786416707705494-12843-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786416707705494-12843-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.138.195:0
--local_ip_for_outbound_sockets=127.12.138.195
--webserver_interface=127.12.138.195
--webserver_port=0
--tserver_master_addrs=127.12.138.254:39439
--keytab_file=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786416707705494-12843-0/krb5kdc/kudu_127.12.138.195.keytab
--principal=kudu/127.12.138.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.138.212:46727
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786416707705494-12843-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786416707705494-12843-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786416707705494-12843-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 02:52:11.344198 17503 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 02:52:11.344471 17503 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 02:52:11.344515 17503 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 02:52:11.348572 17503 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 02:52:11.348694 17503 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 02:52:11.348820 17503 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.138.195
I20260811 02:52:11.354035 17503 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.138.212:46727
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786416707705494-12843-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786416707705494-12843-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786416707705494-12843-0/krb5kdc/kudu_127.12.138.195.keytab
--principal=kudu/127.12.138.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.138.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786416707705494-12843-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.12.138.195
--webserver_port=0
--tserver_master_addrs=127.12.138.254:39439
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17503
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.138.195
--log_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786416707705494-12843-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 02:09:49 UTC on ccycloud.quasar-fsnqqc.root.comops.site
build id 4579570
I20260811 02:52:11.355118 17503 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 02:52:11.356173 17503 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 02:52:11.365814 17512 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:52:11.365998 17503 server_base.cc:1034] running on GCE node
W20260811 02:52:11.365854 17509 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:52:11.365814 17510 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:52:11.366559 17503 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 02:52:11.367224 17503 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 02:52:11.368427 17503 hybrid_clock.cc:648] HybridClock initialized: now 1786416731368390 us; error 46 us; skew 500 ppm
Aug 11 02:52:11 dist-test-slave-9gcw krb5kdc[17055](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416731, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.138.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 02:52:11.371246 17503 init.cc:377] Logged in from keytab as kudu/127.12.138.195@KRBTEST.COM (short username kudu)
I20260811 02:52:11.372193 17503 webserver.cc:460] Webserver started at http://127.12.138.195:38769/ using document root <none> and password file <none>
I20260811 02:52:11.372764 17503 fs_manager.cc:359] Metadata directory not provided
I20260811 02:52:11.372860 17503 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 02:52:11.373104 17503 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 02:52:11.375252 17503 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786416707705494-12843-0/minicluster-data/ts-2/data/instance:
uuid: "a93e92c806df4e32a244e56d0603465e"
format_stamp: "Formatted at 2026-08-11 02:52:11 on dist-test-slave-9gcw"
server_key: "0a50c7b563b0c9a0c84a81b531edd948"
server_key_iv: "d819903eae5afac91ce02830e9199eb8"
server_key_version: "encryptionkey@0"
I20260811 02:52:11.375869 17503 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786416707705494-12843-0/minicluster-data/ts-2/wal/instance:
uuid: "a93e92c806df4e32a244e56d0603465e"
format_stamp: "Formatted at 2026-08-11 02:52:11 on dist-test-slave-9gcw"
server_key: "0a50c7b563b0c9a0c84a81b531edd948"
server_key_iv: "d819903eae5afac91ce02830e9199eb8"
server_key_version: "encryptionkey@0"
I20260811 02:52:11.379757 17503 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.003s
I20260811 02:52:11.382515 17522 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 02:52:11.383980 17503 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260811 02:52:11.384130 17503 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786416707705494-12843-0/minicluster-data/ts-2/data,/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786416707705494-12843-0/minicluster-data/ts-2/wal
uuid: "a93e92c806df4e32a244e56d0603465e"
format_stamp: "Formatted at 2026-08-11 02:52:11 on dist-test-slave-9gcw"
server_key: "0a50c7b563b0c9a0c84a81b531edd948"
server_key_iv: "d819903eae5afac91ce02830e9199eb8"
server_key_version: "encryptionkey@0"
I20260811 02:52:11.384253 17503 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786416707705494-12843-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786416707705494-12843-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786416707705494-12843-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 02:52:11.421253 17503 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 02:52:11.424551 17503 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 02:52:11.424782 17503 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 02:52:11.425431 17503 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 02:52:11.426366 17503 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 02:52:11.426440 17503 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 02:52:11.426507 17503 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 02:52:11.426559 17503 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 02:52:11.436259 17503 rpc_server.cc:307] RPC server started. Bound to: 127.12.138.195:35769
I20260811 02:52:11.436316 17663 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.138.195:35769 every 8 connection(s)
I20260811 02:52:11.437271 17503 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786416707705494-12843-0/minicluster-data/ts-2/data/info.pb
I20260811 02:52:11.440383 12843 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDawEwS/build/debug/bin/kudu as pid 17503
I20260811 02:52:11.440539 12843 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786416707705494-12843-0/minicluster-data/ts-2/wal/instance
I20260811 02:52:11.440857 12843 external_mini_cluster.cc:1428] Setting key 207aed9f499ae38ae260ab9f1bc7f362
Aug 11 02:52:11 dist-test-slave-9gcw krb5kdc[17055](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416731, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.138.195@KRBTEST.COM for kudu/127.12.138.254@KRBTEST.COM
I20260811 02:52:11.452973 17156 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:52:11.439098 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.138.195:40569 (local address 127.12.138.254:39439)
0811 02:52:11.439266 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0811 02:52:11.439271 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 02:52:11.440174 (+ 903us) server_negotiation.cc:407] Connection header received
0811 02:52:11.442316 (+ 2142us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:52:11.442319 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:52:11.442374 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:52:11.442460 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:52:11.444061 (+ 1601us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:52:11.444612 (+ 551us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:52:11.446020 (+ 1408us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:52:11.446248 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:52:11.446340 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:52:11.449203 (+ 2863us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:52:11.449213 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:52:11.449215 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:52:11.449243 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:52:11.450985 (+ 1742us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:52:11.451493 (+ 508us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:52:11.451496 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:52:11.451497 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:52:11.451575 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:52:11.451925 (+ 350us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:52:11.451927 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:52:11.451928 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:52:11.452112 (+ 184us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:52:11.452191 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0811 02:52:11.452719 (+ 528us) server_negotiation.cc:299] Negotiation successful
0811 02:52:11.452835 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260811 02:52:11.453934 17667 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:52:11.439517 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.138.254:39439 (local address 127.12.138.195:40569)
0811 02:52:11.439990 (+ 473us) negotiation.cc:107] Waiting for socket to connect
0811 02:52:11.440037 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0811 02:52:11.442015 (+ 1978us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:52:11.442713 (+ 698us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:52:11.442724 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:52:11.443300 (+ 576us) client_negotiation.cc:189] Negotiated authn=SASL
0811 02:52:11.443873 (+ 573us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:52:11.443889 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:52:11.444762 (+ 873us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:52:11.444765 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:52:11.445891 (+ 1126us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:52:11.445902 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:52:11.446395 (+ 493us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:52:11.446398 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:52:11.446558 (+ 160us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:52:11.447196 (+ 638us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 02:52:11.447214 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0811 02:52:11.448979 (+ 1765us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 02:52:11.451100 (+ 2121us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:52:11.451105 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:52:11.451119 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:52:11.451395 (+ 276us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:52:11.451658 (+ 263us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:52:11.451660 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:52:11.451661 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:52:11.451839 (+ 178us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:52:11.452202 (+ 363us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 02:52:11.452207 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 02:52:11.452451 (+ 244us) client_negotiation.cc:769] Sending connection context
0811 02:52:11.452618 (+ 167us) client_negotiation.cc:240] Negotiation successful
0811 02:52:11.452913 (+ 295us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":268,"thread_start_us":116,"threads_started":1}
I20260811 02:52:11.455279 17665 heartbeater.cc:347] Connected to a master server at 127.12.138.254:39439
I20260811 02:52:11.455564 17665 heartbeater.cc:464] Registering TS with master...
I20260811 02:52:11.456099 17665 heartbeater.cc:511] Master 127.12.138.254:39439 requested a full tablet report, sending...
I20260811 02:52:11.457227 17109 ts_manager.cc:194] Registered new tserver with Master: a93e92c806df4e32a244e56d0603465e (127.12.138.195:35769)
I20260811 02:52:11.457837 17109 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.12.138.195@KRBTEST.COM'} at 127.12.138.195:40569
I20260811 02:52:11.467200 12843 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 11 02:52:11 dist-test-slave-9gcw krb5kdc[17055](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416731, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 11 02:52:11 dist-test-slave-9gcw krb5kdc[17055](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416731, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.12.138.193@KRBTEST.COM
I20260811 02:52:11.496779 17672 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:52:11.486702 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46952 (local address 127.12.138.193:38145)
0811 02:52:11.487027 (+ 325us) server_negotiation.cc:206] Beginning negotiation
0811 02:52:11.487037 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 02:52:11.487069 (+ 32us) server_negotiation.cc:407] Connection header received
0811 02:52:11.487124 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:52:11.487129 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:52:11.487323 (+ 194us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:52:11.487476 (+ 153us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:52:11.488224 (+ 748us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:52:11.489087 (+ 863us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:52:11.489957 (+ 870us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:52:11.490328 (+ 371us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:52:11.490410 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:52:11.493084 (+ 2674us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:52:11.493094 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:52:11.493107 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:52:11.493132 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:52:11.494924 (+ 1792us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:52:11.495390 (+ 466us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:52:11.495394 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:52:11.495396 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:52:11.495475 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:52:11.495751 (+ 276us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:52:11.495753 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:52:11.495754 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:52:11.496006 (+ 252us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:52:11.496175 (+ 169us) server_negotiation.cc:1035] Waiting for connection context
0811 02:52:11.496386 (+ 211us) server_negotiation.cc:299] Negotiation successful
0811 02:52:11.496565 (+ 179us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":223,"thread_start_us":151,"threads_started":1}
Aug 11 02:52:11 dist-test-slave-9gcw krb5kdc[17055](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416731, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.12.138.254@KRBTEST.COM
I20260811 02:52:11.508301 17156 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:52:11.499877 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53050 (local address 127.12.138.254:39439)
0811 02:52:11.500048 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0811 02:52:11.500052 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 02:52:11.500066 (+ 14us) server_negotiation.cc:407] Connection header received
0811 02:52:11.500177 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:52:11.500180 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:52:11.500218 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:52:11.500283 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:52:11.500902 (+ 619us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:52:11.501497 (+ 595us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:52:11.502331 (+ 834us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:52:11.502541 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:52:11.502616 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:52:11.505075 (+ 2459us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:52:11.505078 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:52:11.505079 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:52:11.505106 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:52:11.506627 (+ 1521us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:52:11.507026 (+ 399us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:52:11.507029 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:52:11.507030 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:52:11.507104 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:52:11.507416 (+ 312us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:52:11.507418 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:52:11.507419 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:52:11.507638 (+ 219us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:52:11.507741 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0811 02:52:11.507998 (+ 257us) server_negotiation.cc:299] Negotiation successful
0811 02:52:11.508098 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":86}
W20260811 02:52:11.508803 17109 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53050
I20260811 02:52:11.509523 12843 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDawEwS/build/debug/bin/kudu with pid 17181
I20260811 02:52:11.518895 12843 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDawEwS/build/debug/bin/kudu with pid 17338
I20260811 02:52:11.527684 12843 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDawEwS/build/debug/bin/kudu with pid 17503
I20260811 02:52:11.539891 12843 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDawEwS/build/debug/bin/kudu with pid 17068
2026-08-11T02:52:11Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizedSuperuser (2355 ms)
[ RUN ] SecurityITest.TestDisableWebUI
Loading random data
Initializing database '/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786416707705494-12843-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 02:52:11 dist-test-slave-9gcw krb5kdc[17677](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 02:52:11 dist-test-slave-9gcw krb5kdc[17677](info): set up 2 sockets
Aug 11 02:52:11 dist-test-slave-9gcw krb5kdc[17677](info): commencing operation
krb5kdc: starting...
W20260811 02:52:12.734047 12843 mini_kdc.cc:121] Time spent starting KDC: real 1.168s user 0.003s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786416707705494-12843-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786416707705494-12843-0/krb5kdc/test-user.keytab.
Aug 11 02:52:12 dist-test-slave-9gcw krb5kdc[17677](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416732, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T02:52:12Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T02:52:12Z Disabled control of system clock
WARNING: no policy specified for kudu/127.12.138.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.138.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.138.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786416707705494-12843-0/krb5kdc/kudu_127.12.138.254.keytab.
Entry for principal kudu/127.12.138.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786416707705494-12843-0/krb5kdc/kudu_127.12.138.254.keytab.
I20260811 02:52:12.872051 12843 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDawEwS/build/debug/bin/kudu
/tmp/dist-test-taskDawEwS/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786416707705494-12843-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786416707705494-12843-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786416707705494-12843-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786416707705494-12843-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.138.254:45359
--webserver_interface=127.12.138.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786416707705494-12843-0/krb5kdc/kudu_127.12.138.254.keytab
--principal=kudu/127.12.138.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.138.212:35443
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.12.138.254:45359
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786416707705494-12843-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786416707705494-12843-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786416707705494-12843-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 02:52:12.993333 17691 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 02:52:12.993781 17691 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 02:52:12.993877 17691 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 02:52:12.998185 17691 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 02:52:12.998272 17691 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 02:52:12.998308 17691 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 02:52:12.998348 17691 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 02:52:12.998380 17691 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 02:52:13.004077 17691 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.138.212:35443
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786416707705494-12843-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786416707705494-12843-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.12.138.254:45359
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786416707705494-12843-0/krb5kdc/kudu_127.12.138.254.keytab
--principal=kudu/127.12.138.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.138.254:45359
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786416707705494-12843-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.12.138.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17691
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786416707705494-12843-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 02:09:49 UTC on ccycloud.quasar-fsnqqc.root.comops.site
build id 4579570
I20260811 02:52:13.005359 17691 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 02:52:13.006456 17691 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 02:52:13.015686 17703 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:52:13.015738 17701 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:52:13.015722 17700 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:52:13.016062 17691 server_base.cc:1034] running on GCE node
I20260811 02:52:13.016777 17691 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 02:52:13.018003 17691 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 02:52:13.019198 17691 hybrid_clock.cc:648] HybridClock initialized: now 1786416733019167 us; error 48 us; skew 500 ppm
Aug 11 02:52:13 dist-test-slave-9gcw krb5kdc[17677](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416733, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.138.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 02:52:13.028294 17691 init.cc:377] Logged in from keytab as kudu/127.12.138.254@KRBTEST.COM (short username kudu)
I20260811 02:52:13.029275 17691 fs_manager.cc:359] Metadata directory not provided
I20260811 02:52:13.029381 17691 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 02:52:13.029623 17691 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 02:52:13.031841 17691 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786416707705494-12843-0/minicluster-data/master-0/data/instance:
uuid: "30098621f7d7498b8e58014c89f48ed3"
format_stamp: "Formatted at 2026-08-11 02:52:13 on dist-test-slave-9gcw"
server_key: "254ca2360f51beed5e872578f1f082d7"
server_key_iv: "b1db98647ca095fc3f3a3060838a56dd"
server_key_version: "encryptionkey@0"
I20260811 02:52:13.032447 17691 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786416707705494-12843-0/minicluster-data/master-0/wal/instance:
uuid: "30098621f7d7498b8e58014c89f48ed3"
format_stamp: "Formatted at 2026-08-11 02:52:13 on dist-test-slave-9gcw"
server_key: "254ca2360f51beed5e872578f1f082d7"
server_key_iv: "b1db98647ca095fc3f3a3060838a56dd"
server_key_version: "encryptionkey@0"
I20260811 02:52:13.036480 17691 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260811 02:52:13.039110 17714 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 02:52:13.040553 17691 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260811 02:52:13.040704 17691 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786416707705494-12843-0/minicluster-data/master-0/data,/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786416707705494-12843-0/minicluster-data/master-0/wal
uuid: "30098621f7d7498b8e58014c89f48ed3"
format_stamp: "Formatted at 2026-08-11 02:52:13 on dist-test-slave-9gcw"
server_key: "254ca2360f51beed5e872578f1f082d7"
server_key_iv: "b1db98647ca095fc3f3a3060838a56dd"
server_key_version: "encryptionkey@0"
I20260811 02:52:13.040827 17691 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786416707705494-12843-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786416707705494-12843-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786416707705494-12843-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 02:52:13.064751 17691 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 02:52:13.068315 17691 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 02:52:13.068547 17691 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 02:52:13.076222 17691 rpc_server.cc:307] RPC server started. Bound to: 127.12.138.254:45359
I20260811 02:52:13.076534 17774 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.138.254:45359 every 8 connection(s)
I20260811 02:52:13.076994 17691 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786416707705494-12843-0/minicluster-data/master-0/data/info.pb
I20260811 02:52:13.080122 17775 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 02:52:13.081336 12843 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDawEwS/build/debug/bin/kudu as pid 17691
I20260811 02:52:13.081492 12843 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786416707705494-12843-0/minicluster-data/master-0/wal/instance
I20260811 02:52:13.081881 12843 external_mini_cluster.cc:1428] Setting key 0f66881c257b94c774ad0f52dbdaa8fd
I20260811 02:52:13.087364 17775 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 30098621f7d7498b8e58014c89f48ed3: Bootstrap starting.
I20260811 02:52:13.089864 17775 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 30098621f7d7498b8e58014c89f48ed3: Neither blocks nor log segments found. Creating new log.
I20260811 02:52:13.090782 17775 log.cc:824] T 00000000000000000000000000000000 P 30098621f7d7498b8e58014c89f48ed3: Log is configured to *not* fsync() on all Append() calls
Aug 11 02:52:13 dist-test-slave-9gcw krb5kdc[17677](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416732, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.12.138.254@KRBTEST.COM
I20260811 02:52:13.093231 17775 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 30098621f7d7498b8e58014c89f48ed3: No bootstrap required, opened a new log
I20260811 02:52:13.096865 17778 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:52:13.083543 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57818 (local address 127.12.138.254:45359)
0811 02:52:13.084024 (+ 481us) server_negotiation.cc:206] Beginning negotiation
0811 02:52:13.084040 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0811 02:52:13.084075 (+ 35us) server_negotiation.cc:407] Connection header received
0811 02:52:13.084708 (+ 633us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:52:13.084726 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:52:13.085067 (+ 341us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:52:13.085492 (+ 425us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:52:13.086295 (+ 803us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:52:13.087279 (+ 984us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:52:13.088206 (+ 927us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:52:13.088695 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:52:13.088872 (+ 177us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:52:13.091824 (+ 2952us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:52:13.091829 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:52:13.091843 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:52:13.091868 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:52:13.093755 (+ 1887us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:52:13.094301 (+ 546us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:52:13.094306 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:52:13.094310 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:52:13.094418 (+ 108us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:52:13.094734 (+ 316us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:52:13.094737 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:52:13.094738 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:52:13.095204 (+ 466us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:52:13.095369 (+ 165us) server_negotiation.cc:1035] Waiting for connection context
0811 02:52:13.095769 (+ 400us) server_negotiation.cc:299] Negotiation successful
0811 02:52:13.096068 (+ 299us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":335,"thread_start_us":156,"threads_started":1}
I20260811 02:52:13.097801 17775 raft_consensus.cc:348] T 00000000000000000000000000000000 P 30098621f7d7498b8e58014c89f48ed3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "30098621f7d7498b8e58014c89f48ed3" member_type: VOTER last_known_addr { host: "127.12.138.254" port: 45359 } }
I20260811 02:52:13.098043 17775 raft_consensus.cc:374] T 00000000000000000000000000000000 P 30098621f7d7498b8e58014c89f48ed3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 02:52:13.098130 17775 raft_consensus.cc:729] T 00000000000000000000000000000000 P 30098621f7d7498b8e58014c89f48ed3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 30098621f7d7498b8e58014c89f48ed3, State: Initialized, Role: FOLLOWER
I20260811 02:52:13.098563 17775 consensus_queue.cc:260] T 00000000000000000000000000000000 P 30098621f7d7498b8e58014c89f48ed3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "30098621f7d7498b8e58014c89f48ed3" member_type: VOTER last_known_addr { host: "127.12.138.254" port: 45359 } }
I20260811 02:52:13.098701 17775 raft_consensus.cc:388] T 00000000000000000000000000000000 P 30098621f7d7498b8e58014c89f48ed3 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 02:52:13.098796 17775 raft_consensus.cc:482] T 00000000000000000000000000000000 P 30098621f7d7498b8e58014c89f48ed3 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 02:52:13.098935 17775 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 30098621f7d7498b8e58014c89f48ed3 [term 0 FOLLOWER]: Advancing to term 1
I20260811 02:52:13.099973 17775 raft_consensus.cc:504] T 00000000000000000000000000000000 P 30098621f7d7498b8e58014c89f48ed3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "30098621f7d7498b8e58014c89f48ed3" member_type: VOTER last_known_addr { host: "127.12.138.254" port: 45359 } }
I20260811 02:52:13.100338 17775 leader_election.cc:302] T 00000000000000000000000000000000 P 30098621f7d7498b8e58014c89f48ed3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 30098621f7d7498b8e58014c89f48ed3; no voters:
I20260811 02:52:13.100780 17775 leader_election.cc:288] T 00000000000000000000000000000000 P 30098621f7d7498b8e58014c89f48ed3 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 02:52:13.100874 17780 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 30098621f7d7498b8e58014c89f48ed3 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 02:52:13.101147 17780 raft_consensus.cc:686] T 00000000000000000000000000000000 P 30098621f7d7498b8e58014c89f48ed3 [term 1 LEADER]: Becoming Leader. State: Replica: 30098621f7d7498b8e58014c89f48ed3, State: Running, Role: LEADER
I20260811 02:52:13.101516 17780 consensus_queue.cc:237] T 00000000000000000000000000000000 P 30098621f7d7498b8e58014c89f48ed3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "30098621f7d7498b8e58014c89f48ed3" member_type: VOTER last_known_addr { host: "127.12.138.254" port: 45359 } }
I20260811 02:52:13.102308 17775 sys_catalog.cc:564] T 00000000000000000000000000000000 P 30098621f7d7498b8e58014c89f48ed3 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 02:52:13.103453 17781 sys_catalog.cc:455] T 00000000000000000000000000000000 P 30098621f7d7498b8e58014c89f48ed3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "30098621f7d7498b8e58014c89f48ed3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "30098621f7d7498b8e58014c89f48ed3" member_type: VOTER last_known_addr { host: "127.12.138.254" port: 45359 } } }
I20260811 02:52:13.103442 17782 sys_catalog.cc:455] T 00000000000000000000000000000000 P 30098621f7d7498b8e58014c89f48ed3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 30098621f7d7498b8e58014c89f48ed3. Latest consensus state: current_term: 1 leader_uuid: "30098621f7d7498b8e58014c89f48ed3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "30098621f7d7498b8e58014c89f48ed3" member_type: VOTER last_known_addr { host: "127.12.138.254" port: 45359 } } }
I20260811 02:52:13.103641 17781 sys_catalog.cc:458] T 00000000000000000000000000000000 P 30098621f7d7498b8e58014c89f48ed3 [sys.catalog]: This master's current role is: LEADER
I20260811 02:52:13.103648 17782 sys_catalog.cc:458] T 00000000000000000000000000000000 P 30098621f7d7498b8e58014c89f48ed3 [sys.catalog]: This master's current role is: LEADER
I20260811 02:52:13.104537 17791 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 02:52:13.107597 17791 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 02:52:13.113412 17791 catalog_manager.cc:1347] Generated new cluster ID: 18bed9636aa54cc19fdfc4a604a51b08
I20260811 02:52:13.113492 17791 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 02:52:13.121685 17791 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 02:52:13.123045 17791 catalog_manager.cc:1504] Loading token signing keys...
I20260811 02:52:13.134910 17791 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 30098621f7d7498b8e58014c89f48ed3: Generated new TSK 0
I20260811 02:52:13.135710 17791 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.12.138.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.138.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.138.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786416707705494-12843-0/krb5kdc/kudu_127.12.138.193.keytab.
Entry for principal kudu/127.12.138.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786416707705494-12843-0/krb5kdc/kudu_127.12.138.193.keytab.
I20260811 02:52:13.174662 12843 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDawEwS/build/debug/bin/kudu
/tmp/dist-test-taskDawEwS/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786416707705494-12843-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786416707705494-12843-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786416707705494-12843-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786416707705494-12843-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.138.193:0
--local_ip_for_outbound_sockets=127.12.138.193
--webserver_interface=127.12.138.193
--webserver_port=0
--tserver_master_addrs=127.12.138.254:45359
--keytab_file=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786416707705494-12843-0/krb5kdc/kudu_127.12.138.193.keytab
--principal=kudu/127.12.138.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.138.212:35443
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786416707705494-12843-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786416707705494-12843-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786416707705494-12843-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 02:52:13.294219 17804 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 02:52:13.294530 17804 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 02:52:13.294618 17804 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 02:52:13.298925 17804 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 02:52:13.299021 17804 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 02:52:13.299120 17804 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.138.193
I20260811 02:52:13.304749 17804 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.138.212:35443
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786416707705494-12843-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786416707705494-12843-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786416707705494-12843-0/krb5kdc/kudu_127.12.138.193.keytab
--principal=kudu/127.12.138.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.138.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786416707705494-12843-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.12.138.193
--webserver_port=0
--tserver_master_addrs=127.12.138.254:45359
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17804
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.138.193
--log_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786416707705494-12843-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 02:09:49 UTC on ccycloud.quasar-fsnqqc.root.comops.site
build id 4579570
I20260811 02:52:13.305995 17804 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 02:52:13.307066 17804 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 02:52:13.317312 17813 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:52:13.317306 17811 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:52:13.317301 17810 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:52:13.317790 17804 server_base.cc:1034] running on GCE node
I20260811 02:52:13.318193 17804 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 02:52:13.318821 17804 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 02:52:13.320024 17804 hybrid_clock.cc:648] HybridClock initialized: now 1786416733320010 us; error 25 us; skew 500 ppm
Aug 11 02:52:13 dist-test-slave-9gcw krb5kdc[17677](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416733, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.138.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 02:52:13.323201 17804 init.cc:377] Logged in from keytab as kudu/127.12.138.193@KRBTEST.COM (short username kudu)
I20260811 02:52:13.323953 17804 fs_manager.cc:359] Metadata directory not provided
I20260811 02:52:13.324050 17804 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 02:52:13.324296 17804 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 02:52:13.326470 17804 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786416707705494-12843-0/minicluster-data/ts-0/data/instance:
uuid: "0d07fa6d87d04eb9b0af513a2a53f8f5"
format_stamp: "Formatted at 2026-08-11 02:52:13 on dist-test-slave-9gcw"
server_key: "e87f01344a80e0dfd9932728645e886e"
server_key_iv: "d474551c34c78865fad2ec2671eef31e"
server_key_version: "encryptionkey@0"
I20260811 02:52:13.327071 17804 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786416707705494-12843-0/minicluster-data/ts-0/wal/instance:
uuid: "0d07fa6d87d04eb9b0af513a2a53f8f5"
format_stamp: "Formatted at 2026-08-11 02:52:13 on dist-test-slave-9gcw"
server_key: "e87f01344a80e0dfd9932728645e886e"
server_key_iv: "d474551c34c78865fad2ec2671eef31e"
server_key_version: "encryptionkey@0"
I20260811 02:52:13.331079 17804 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260811 02:52:13.333648 17819 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 02:52:13.334909 17804 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260811 02:52:13.335048 17804 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786416707705494-12843-0/minicluster-data/ts-0/data,/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786416707705494-12843-0/minicluster-data/ts-0/wal
uuid: "0d07fa6d87d04eb9b0af513a2a53f8f5"
format_stamp: "Formatted at 2026-08-11 02:52:13 on dist-test-slave-9gcw"
server_key: "e87f01344a80e0dfd9932728645e886e"
server_key_iv: "d474551c34c78865fad2ec2671eef31e"
server_key_version: "encryptionkey@0"
I20260811 02:52:13.335168 17804 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786416707705494-12843-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786416707705494-12843-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786416707705494-12843-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 02:52:13.357862 17804 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 02:52:13.361229 17804 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 02:52:13.361436 17804 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 02:52:13.362076 17804 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 02:52:13.363031 17804 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 02:52:13.363104 17804 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 02:52:13.363168 17804 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 02:52:13.363220 17804 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 02:52:13.372427 17804 rpc_server.cc:307] RPC server started. Bound to: 127.12.138.193:32807
I20260811 02:52:13.372447 17960 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.138.193:32807 every 8 connection(s)
I20260811 02:52:13.373049 17804 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786416707705494-12843-0/minicluster-data/ts-0/data/info.pb
I20260811 02:52:13.382921 12843 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDawEwS/build/debug/bin/kudu as pid 17804
I20260811 02:52:13.383035 12843 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786416707705494-12843-0/minicluster-data/ts-0/wal/instance
I20260811 02:52:13.383337 12843 external_mini_cluster.cc:1428] Setting key c2552b1e60aacaf5f3b90d024e74a244
Aug 11 02:52:13 dist-test-slave-9gcw krb5kdc[17677](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416733, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.138.193@KRBTEST.COM for kudu/127.12.138.254@KRBTEST.COM
I20260811 02:52:13.388567 17778 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:52:13.374940 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.138.193:39675 (local address 127.12.138.254:45359)
0811 02:52:13.375114 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0811 02:52:13.375119 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 02:52:13.376066 (+ 947us) server_negotiation.cc:407] Connection header received
0811 02:52:13.377625 (+ 1559us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:52:13.377628 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:52:13.377676 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:52:13.377757 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:52:13.379040 (+ 1283us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:52:13.379994 (+ 954us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:52:13.381273 (+ 1279us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:52:13.381506 (+ 233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:52:13.381590 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:52:13.384503 (+ 2913us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:52:13.384507 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:52:13.384510 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:52:13.384534 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:52:13.386362 (+ 1828us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:52:13.386943 (+ 581us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:52:13.386947 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:52:13.386948 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:52:13.387029 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:52:13.387399 (+ 370us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:52:13.387404 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:52:13.387406 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:52:13.387596 (+ 190us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:52:13.387710 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0811 02:52:13.388265 (+ 555us) server_negotiation.cc:299] Negotiation successful
0811 02:52:13.388397 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260811 02:52:13.389487 17963 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:52:13.375334 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.138.254:45359 (local address 127.12.138.193:39675)
0811 02:52:13.375884 (+ 550us) negotiation.cc:107] Waiting for socket to connect
0811 02:52:13.375932 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0811 02:52:13.377406 (+ 1474us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:52:13.377952 (+ 546us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:52:13.377959 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:52:13.378315 (+ 356us) client_negotiation.cc:189] Negotiated authn=SASL
0811 02:52:13.378892 (+ 577us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:52:13.378909 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:52:13.380141 (+ 1232us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:52:13.380145 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:52:13.381067 (+ 922us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:52:13.381077 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:52:13.381655 (+ 578us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:52:13.381658 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:52:13.381821 (+ 163us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:52:13.382474 (+ 653us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 02:52:13.382492 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0811 02:52:13.384336 (+ 1844us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 02:52:13.386498 (+ 2162us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:52:13.386505 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:52:13.386522 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:52:13.386815 (+ 293us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:52:13.387142 (+ 327us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:52:13.387145 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:52:13.387147 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:52:13.387296 (+ 149us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:52:13.387725 (+ 429us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 02:52:13.387734 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 02:52:13.388000 (+ 266us) client_negotiation.cc:769] Sending connection context
0811 02:52:13.388207 (+ 207us) client_negotiation.cc:240] Negotiation successful
0811 02:52:13.388549 (+ 342us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":302,"thread_start_us":114,"threads_started":1}
I20260811 02:52:13.390878 17961 heartbeater.cc:347] Connected to a master server at 127.12.138.254:45359
I20260811 02:52:13.391144 17961 heartbeater.cc:464] Registering TS with master...
I20260811 02:52:13.391628 17961 heartbeater.cc:511] Master 127.12.138.254:45359 requested a full tablet report, sending...
I20260811 02:52:13.393399 17733 ts_manager.cc:194] Registered new tserver with Master: 0d07fa6d87d04eb9b0af513a2a53f8f5 (127.12.138.193:32807)
WARNING: no policy specified for kudu/127.12.138.194@KRBTEST.COM; defaulting to no policy
I20260811 02:52:13.394613 17733 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.12.138.193@KRBTEST.COM'} at 127.12.138.193:39675
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.138.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.138.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786416707705494-12843-0/krb5kdc/kudu_127.12.138.194.keytab.
Entry for principal kudu/127.12.138.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786416707705494-12843-0/krb5kdc/kudu_127.12.138.194.keytab.
I20260811 02:52:13.414669 12843 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDawEwS/build/debug/bin/kudu
/tmp/dist-test-taskDawEwS/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786416707705494-12843-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786416707705494-12843-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786416707705494-12843-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786416707705494-12843-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.138.194:0
--local_ip_for_outbound_sockets=127.12.138.194
--webserver_interface=127.12.138.194
--webserver_port=0
--tserver_master_addrs=127.12.138.254:45359
--keytab_file=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786416707705494-12843-0/krb5kdc/kudu_127.12.138.194.keytab
--principal=kudu/127.12.138.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.138.212:35443
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786416707705494-12843-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786416707705494-12843-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786416707705494-12843-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 02:52:13.529608 17967 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 02:52:13.529878 17967 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 02:52:13.529954 17967 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 02:52:13.533977 17967 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 02:52:13.534039 17967 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 02:52:13.534130 17967 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.138.194
I20260811 02:52:13.539569 17967 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.138.212:35443
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786416707705494-12843-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786416707705494-12843-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786416707705494-12843-0/krb5kdc/kudu_127.12.138.194.keytab
--principal=kudu/127.12.138.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.138.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786416707705494-12843-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.12.138.194
--webserver_port=0
--tserver_master_addrs=127.12.138.254:45359
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17967
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.138.194
--log_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786416707705494-12843-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 02:09:49 UTC on ccycloud.quasar-fsnqqc.root.comops.site
build id 4579570
I20260811 02:52:13.540813 17967 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 02:52:13.541956 17967 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 02:52:13.551668 17974 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:52:13.551699 17973 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:52:13.551668 17977 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:52:13.552096 17967 server_base.cc:1034] running on GCE node
I20260811 02:52:13.552570 17967 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 02:52:13.553323 17967 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 02:52:13.554508 17967 hybrid_clock.cc:648] HybridClock initialized: now 1786416733554471 us; error 45 us; skew 500 ppm
Aug 11 02:52:13 dist-test-slave-9gcw krb5kdc[17677](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416733, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.138.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 02:52:13.557536 17967 init.cc:377] Logged in from keytab as kudu/127.12.138.194@KRBTEST.COM (short username kudu)
I20260811 02:52:13.558323 17967 fs_manager.cc:359] Metadata directory not provided
I20260811 02:52:13.558423 17967 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 02:52:13.558673 17967 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 02:52:13.560746 17967 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786416707705494-12843-0/minicluster-data/ts-1/data/instance:
uuid: "e06050d162db48209c6e3739dd9db64e"
format_stamp: "Formatted at 2026-08-11 02:52:13 on dist-test-slave-9gcw"
server_key: "65c6ac90ed81c279370503e33b9e330f"
server_key_iv: "99626442b8419a2a9c233178abea9751"
server_key_version: "encryptionkey@0"
I20260811 02:52:13.561386 17967 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786416707705494-12843-0/minicluster-data/ts-1/wal/instance:
uuid: "e06050d162db48209c6e3739dd9db64e"
format_stamp: "Formatted at 2026-08-11 02:52:13 on dist-test-slave-9gcw"
server_key: "65c6ac90ed81c279370503e33b9e330f"
server_key_iv: "99626442b8419a2a9c233178abea9751"
server_key_version: "encryptionkey@0"
I20260811 02:52:13.565434 17967 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260811 02:52:13.568125 17984 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 02:52:13.569550 17967 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260811 02:52:13.569698 17967 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786416707705494-12843-0/minicluster-data/ts-1/data,/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786416707705494-12843-0/minicluster-data/ts-1/wal
uuid: "e06050d162db48209c6e3739dd9db64e"
format_stamp: "Formatted at 2026-08-11 02:52:13 on dist-test-slave-9gcw"
server_key: "65c6ac90ed81c279370503e33b9e330f"
server_key_iv: "99626442b8419a2a9c233178abea9751"
server_key_version: "encryptionkey@0"
I20260811 02:52:13.569823 17967 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786416707705494-12843-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786416707705494-12843-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786416707705494-12843-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 02:52:13.583328 17967 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 02:52:13.586536 17967 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 02:52:13.586768 17967 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 02:52:13.587419 17967 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 02:52:13.588320 17967 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 02:52:13.588411 17967 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 02:52:13.588526 17967 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 02:52:13.588580 17967 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 02:52:13.598713 17967 rpc_server.cc:307] RPC server started. Bound to: 127.12.138.194:34371
I20260811 02:52:13.598711 18119 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.138.194:34371 every 8 connection(s)
I20260811 02:52:13.599422 17967 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786416707705494-12843-0/minicluster-data/ts-1/data/info.pb
I20260811 02:52:13.602073 12843 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDawEwS/build/debug/bin/kudu as pid 17967
I20260811 02:52:13.602211 12843 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786416707705494-12843-0/minicluster-data/ts-1/wal/instance
I20260811 02:52:13.602600 12843 external_mini_cluster.cc:1428] Setting key 4fec86bac7abe8531d2f29c911b41925
Aug 11 02:52:13 dist-test-slave-9gcw krb5kdc[17677](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416733, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.138.194@KRBTEST.COM for kudu/127.12.138.254@KRBTEST.COM
I20260811 02:52:13.614593 17778 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:52:13.601282 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.138.194:51299 (local address 127.12.138.254:45359)
0811 02:52:13.601422 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0811 02:52:13.601426 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 02:52:13.603051 (+ 1625us) server_negotiation.cc:407] Connection header received
0811 02:52:13.604149 (+ 1098us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:52:13.604153 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:52:13.604205 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:52:13.604307 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:52:13.605612 (+ 1305us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:52:13.606139 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:52:13.607276 (+ 1137us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:52:13.607501 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:52:13.607612 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:52:13.610464 (+ 2852us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:52:13.610471 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:52:13.610478 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:52:13.610513 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:52:13.612276 (+ 1763us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:52:13.612811 (+ 535us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:52:13.612815 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:52:13.612817 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:52:13.612914 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:52:13.613360 (+ 446us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:52:13.613364 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:52:13.613366 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:52:13.613617 (+ 251us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:52:13.613730 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0811 02:52:13.614263 (+ 533us) server_negotiation.cc:299] Negotiation successful
0811 02:52:13.614405 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
WARNING: no policy specified for kudu/127.12.138.195@KRBTEST.COM; defaulting to no policy
I20260811 02:52:13.615279 18123 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:52:13.601749 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.138.254:45359 (local address 127.12.138.194:51299)
0811 02:52:13.602846 (+ 1097us) negotiation.cc:107] Waiting for socket to connect
0811 02:52:13.602944 (+ 98us) client_negotiation.cc:174] Beginning negotiation
0811 02:52:13.603942 (+ 998us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:52:13.604486 (+ 544us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:52:13.604494 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:52:13.604913 (+ 419us) client_negotiation.cc:189] Negotiated authn=SASL
0811 02:52:13.605472 (+ 559us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:52:13.605486 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:52:13.606265 (+ 779us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:52:13.606269 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:52:13.607151 (+ 882us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:52:13.607160 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:52:13.607650 (+ 490us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:52:13.607654 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:52:13.607823 (+ 169us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:52:13.608485 (+ 662us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 02:52:13.608510 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0811 02:52:13.610286 (+ 1776us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 02:52:13.612400 (+ 2114us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:52:13.612405 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:52:13.612415 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:52:13.612671 (+ 256us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:52:13.613011 (+ 340us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:52:13.613014 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:52:13.613015 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:52:13.613251 (+ 236us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:52:13.613746 (+ 495us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 02:52:13.613752 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 02:52:13.613989 (+ 237us) client_negotiation.cc:769] Sending connection context
0811 02:52:13.614144 (+ 155us) client_negotiation.cc:240] Negotiation successful
0811 02:52:13.614429 (+ 285us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":329,"thread_start_us":154,"threads_started":1}
I20260811 02:52:13.616524 18121 heartbeater.cc:347] Connected to a master server at 127.12.138.254:45359
I20260811 02:52:13.616850 18121 heartbeater.cc:464] Registering TS with master...
I20260811 02:52:13.617431 18121 heartbeater.cc:511] Master 127.12.138.254:45359 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.138.195@KRBTEST.COM" created.
I20260811 02:52:13.618664 17733 ts_manager.cc:194] Registered new tserver with Master: e06050d162db48209c6e3739dd9db64e (127.12.138.194:34371)
I20260811 02:52:13.619534 17733 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.12.138.194@KRBTEST.COM'} at 127.12.138.194:51299
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.138.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786416707705494-12843-0/krb5kdc/kudu_127.12.138.195.keytab.
Entry for principal kudu/127.12.138.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786416707705494-12843-0/krb5kdc/kudu_127.12.138.195.keytab.
I20260811 02:52:13.636605 12843 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDawEwS/build/debug/bin/kudu
/tmp/dist-test-taskDawEwS/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786416707705494-12843-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786416707705494-12843-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786416707705494-12843-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786416707705494-12843-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.138.195:0
--local_ip_for_outbound_sockets=127.12.138.195
--webserver_interface=127.12.138.195
--webserver_port=0
--tserver_master_addrs=127.12.138.254:45359
--keytab_file=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786416707705494-12843-0/krb5kdc/kudu_127.12.138.195.keytab
--principal=kudu/127.12.138.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.138.212:35443
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786416707705494-12843-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786416707705494-12843-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786416707705494-12843-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 02:52:13.751209 18126 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 02:52:13.751497 18126 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 02:52:13.751574 18126 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 02:52:13.755656 18126 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 02:52:13.755744 18126 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 02:52:13.755844 18126 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.138.195
I20260811 02:52:13.761101 18126 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.138.212:35443
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786416707705494-12843-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786416707705494-12843-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786416707705494-12843-0/krb5kdc/kudu_127.12.138.195.keytab
--principal=kudu/127.12.138.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.138.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786416707705494-12843-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.12.138.195
--webserver_port=0
--tserver_master_addrs=127.12.138.254:45359
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18126
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.138.195
--log_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786416707705494-12843-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 02:09:49 UTC on ccycloud.quasar-fsnqqc.root.comops.site
build id 4579570
I20260811 02:52:13.762221 18126 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 02:52:13.763267 18126 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 02:52:13.773124 18131 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:52:13.773123 18132 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:52:13.773270 18134 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:52:13.773600 18126 server_base.cc:1034] running on GCE node
I20260811 02:52:13.774149 18126 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 02:52:13.775027 18126 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 02:52:13.776240 18126 hybrid_clock.cc:648] HybridClock initialized: now 1786416733776206 us; error 55 us; skew 500 ppm
Aug 11 02:52:13 dist-test-slave-9gcw krb5kdc[17677](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416733, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.138.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 02:52:13.779932 18126 init.cc:377] Logged in from keytab as kudu/127.12.138.195@KRBTEST.COM (short username kudu)
I20260811 02:52:13.780800 18126 fs_manager.cc:359] Metadata directory not provided
I20260811 02:52:13.780912 18126 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 02:52:13.781250 18126 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 02:52:13.783624 18126 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786416707705494-12843-0/minicluster-data/ts-2/data/instance:
uuid: "4e614b3483fb4f86aab95f6d9846a5aa"
format_stamp: "Formatted at 2026-08-11 02:52:13 on dist-test-slave-9gcw"
server_key: "5aafc26cc9bc0f6248f122806906f55a"
server_key_iv: "5af9ff15b1f42d9dcf0cf144be360045"
server_key_version: "encryptionkey@0"
I20260811 02:52:13.784281 18126 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786416707705494-12843-0/minicluster-data/ts-2/wal/instance:
uuid: "4e614b3483fb4f86aab95f6d9846a5aa"
format_stamp: "Formatted at 2026-08-11 02:52:13 on dist-test-slave-9gcw"
server_key: "5aafc26cc9bc0f6248f122806906f55a"
server_key_iv: "5af9ff15b1f42d9dcf0cf144be360045"
server_key_version: "encryptionkey@0"
I20260811 02:52:13.788463 18126 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260811 02:52:13.791370 18140 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 02:52:13.792855 18126 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260811 02:52:13.793025 18126 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786416707705494-12843-0/minicluster-data/ts-2/data,/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786416707705494-12843-0/minicluster-data/ts-2/wal
uuid: "4e614b3483fb4f86aab95f6d9846a5aa"
format_stamp: "Formatted at 2026-08-11 02:52:13 on dist-test-slave-9gcw"
server_key: "5aafc26cc9bc0f6248f122806906f55a"
server_key_iv: "5af9ff15b1f42d9dcf0cf144be360045"
server_key_version: "encryptionkey@0"
I20260811 02:52:13.793186 18126 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786416707705494-12843-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786416707705494-12843-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786416707705494-12843-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 02:52:13.813254 18126 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 02:52:13.816879 18126 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 02:52:13.817107 18126 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 02:52:13.817811 18126 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 02:52:13.818706 18126 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 02:52:13.818780 18126 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 02:52:13.818856 18126 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 02:52:13.818907 18126 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 02:52:13.828048 18126 rpc_server.cc:307] RPC server started. Bound to: 127.12.138.195:35605
I20260811 02:52:13.828112 18274 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.138.195:35605 every 8 connection(s)
I20260811 02:52:13.828718 18126 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786416707705494-12843-0/minicluster-data/ts-2/data/info.pb
I20260811 02:52:13.834709 12843 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDawEwS/build/debug/bin/kudu as pid 18126
I20260811 02:52:13.834872 12843 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786416707705494-12843-0/minicluster-data/ts-2/wal/instance
I20260811 02:52:13.835233 12843 external_mini_cluster.cc:1428] Setting key 7085e846e396254862db08aa432cdf70
Aug 11 02:52:13 dist-test-slave-9gcw krb5kdc[17677](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416733, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.138.195@KRBTEST.COM for kudu/127.12.138.254@KRBTEST.COM
I20260811 02:52:13.844139 17778 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:52:13.830605 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.138.195:50887 (local address 127.12.138.254:45359)
0811 02:52:13.830809 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0811 02:52:13.830815 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 02:52:13.831738 (+ 923us) server_negotiation.cc:407] Connection header received
0811 02:52:13.833021 (+ 1283us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:52:13.833024 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:52:13.833127 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:52:13.833280 (+ 153us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:52:13.834554 (+ 1274us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:52:13.835388 (+ 834us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:52:13.836820 (+ 1432us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:52:13.837201 (+ 381us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:52:13.837320 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:52:13.840088 (+ 2768us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:52:13.840091 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:52:13.840093 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:52:13.840134 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:52:13.841778 (+ 1644us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:52:13.842348 (+ 570us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:52:13.842354 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:52:13.842356 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:52:13.842448 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:52:13.842864 (+ 416us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:52:13.842870 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:52:13.842875 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:52:13.843088 (+ 213us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:52:13.843211 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0811 02:52:13.843805 (+ 594us) server_negotiation.cc:299] Negotiation successful
0811 02:52:13.843943 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":87}
I20260811 02:52:13.845008 18277 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:52:13.831101 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.138.254:45359 (local address 127.12.138.195:50887)
0811 02:52:13.831578 (+ 477us) negotiation.cc:107] Waiting for socket to connect
0811 02:52:13.831620 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0811 02:52:13.832777 (+ 1157us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:52:13.833491 (+ 714us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:52:13.833500 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:52:13.833904 (+ 404us) client_negotiation.cc:189] Negotiated authn=SASL
0811 02:52:13.834393 (+ 489us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:52:13.834408 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:52:13.835520 (+ 1112us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:52:13.835523 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:52:13.836683 (+ 1160us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:52:13.836694 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:52:13.837345 (+ 651us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:52:13.837348 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:52:13.837492 (+ 144us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:52:13.838148 (+ 656us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 02:52:13.838164 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0811 02:52:13.839837 (+ 1673us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 02:52:13.841915 (+ 2078us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:52:13.841921 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:52:13.841938 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:52:13.842234 (+ 296us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:52:13.842560 (+ 326us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:52:13.842563 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:52:13.842566 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:52:13.842755 (+ 189us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:52:13.843224 (+ 469us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 02:52:13.843233 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 02:52:13.843507 (+ 274us) client_negotiation.cc:769] Sending connection context
0811 02:52:13.843733 (+ 226us) client_negotiation.cc:240] Negotiation successful
0811 02:52:13.844112 (+ 379us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":282,"thread_start_us":127,"threads_started":1}
I20260811 02:52:13.846257 18275 heartbeater.cc:347] Connected to a master server at 127.12.138.254:45359
I20260811 02:52:13.846486 18275 heartbeater.cc:464] Registering TS with master...
I20260811 02:52:13.846971 18275 heartbeater.cc:511] Master 127.12.138.254:45359 requested a full tablet report, sending...
I20260811 02:52:13.848048 17733 ts_manager.cc:194] Registered new tserver with Master: 4e614b3483fb4f86aab95f6d9846a5aa (127.12.138.195:35605)
I20260811 02:52:13.848716 17733 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.12.138.195@KRBTEST.COM'} at 127.12.138.195:50887
I20260811 02:52:13.849748 12843 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 02:52:13.859592 17778 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:52:13.851725 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57834 (local address 127.12.138.254:45359)
0811 02:52:13.851847 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0811 02:52:13.851853 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 02:52:13.852063 (+ 210us) server_negotiation.cc:407] Connection header received
0811 02:52:13.852212 (+ 149us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:52:13.852215 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:52:13.852258 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:52:13.852360 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:52:13.853077 (+ 717us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:52:13.853676 (+ 599us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:52:13.854529 (+ 853us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:52:13.854764 (+ 235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:52:13.854875 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:52:13.856277 (+ 1402us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:52:13.856280 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:52:13.856283 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:52:13.856317 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:52:13.857798 (+ 1481us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:52:13.858223 (+ 425us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:52:13.858228 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:52:13.858230 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:52:13.858327 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:52:13.858608 (+ 281us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:52:13.858613 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:52:13.858614 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:52:13.858851 (+ 237us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:52:13.858996 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0811 02:52:13.859303 (+ 307us) server_negotiation.cc:299] Negotiation successful
0811 02:52:13.859427 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260811 02:52:13.862519 17733 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57834:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 02:52:13.866181 17733 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 02:52:13.881690 18293 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:52:13.876101 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40070 (local address 127.12.138.193:32807)
0811 02:52:13.876432 (+ 331us) server_negotiation.cc:206] Beginning negotiation
0811 02:52:13.876440 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 02:52:13.876492 (+ 52us) server_negotiation.cc:407] Connection header received
0811 02:52:13.876667 (+ 175us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:52:13.876675 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:52:13.876892 (+ 217us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:52:13.877051 (+ 159us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 02:52:13.877422 (+ 371us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:52:13.878292 (+ 870us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:52:13.880009 (+ 1717us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:52:13.880843 (+ 834us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:52:13.880979 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:52:13.881125 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0811 02:52:13.881446 (+ 321us) server_negotiation.cc:299] Negotiation successful
0811 02:52:13.881545 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":220,"thread_start_us":109,"threads_started":1}
I20260811 02:52:13.881650 18288 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:52:13.875288 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.138.194:34371 (local address 127.0.0.1:50366)
0811 02:52:13.875813 (+ 525us) negotiation.cc:107] Waiting for socket to connect
0811 02:52:13.875865 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0811 02:52:13.876007 (+ 142us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:52:13.877344 (+ 1337us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:52:13.877351 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:52:13.877374 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 02:52:13.877489 (+ 115us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:52:13.877497 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:52:13.879018 (+ 1521us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:52:13.879022 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:52:13.880191 (+ 1169us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:52:13.880199 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:52:13.881134 (+ 935us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:52:13.881137 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:52:13.881240 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:52:13.881257 (+ 17us) client_negotiation.cc:769] Sending connection context
0811 02:52:13.881318 (+ 61us) client_negotiation.cc:240] Negotiation successful
0811 02:52:13.881476 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":328,"mutex_wait_us":65,"thread_start_us":77,"threads_started":1}
I20260811 02:52:13.881672 18294 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:52:13.876188 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57852 (local address 127.12.138.195:35605)
0811 02:52:13.876958 (+ 770us) server_negotiation.cc:206] Beginning negotiation
0811 02:52:13.876972 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0811 02:52:13.876996 (+ 24us) server_negotiation.cc:407] Connection header received
0811 02:52:13.877066 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:52:13.877071 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:52:13.877297 (+ 226us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:52:13.877440 (+ 143us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 02:52:13.877776 (+ 336us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:52:13.878887 (+ 1111us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:52:13.880310 (+ 1423us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:52:13.881104 (+ 794us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:52:13.881255 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:52:13.881389 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0811 02:52:13.881442 (+ 53us) server_negotiation.cc:299] Negotiation successful
0811 02:52:13.881531 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":673,"thread_start_us":126,"threads_started":1}
I20260811 02:52:13.881656 18292 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:52:13.876024 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.138.195:35605 (local address 127.0.0.1:57852)
0811 02:52:13.876607 (+ 583us) negotiation.cc:107] Waiting for socket to connect
0811 02:52:13.876621 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 02:52:13.876709 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:52:13.877537 (+ 828us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:52:13.877540 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:52:13.877557 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 02:52:13.877645 (+ 88us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:52:13.877651 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:52:13.878998 (+ 1347us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:52:13.879001 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:52:13.880157 (+ 1156us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:52:13.880167 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:52:13.881239 (+ 1072us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:52:13.881243 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:52:13.881327 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:52:13.881350 (+ 23us) client_negotiation.cc:769] Sending connection context
0811 02:52:13.881417 (+ 67us) client_negotiation.cc:240] Negotiation successful
0811 02:52:13.881499 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":495,"mutex_wait_us":40,"thread_start_us":84,"threads_started":1}
I20260811 02:52:13.882313 18291 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:52:13.875436 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.138.193:32807 (local address 127.0.0.1:40070)
0811 02:52:13.876400 (+ 964us) negotiation.cc:107] Waiting for socket to connect
0811 02:52:13.876414 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 02:52:13.876524 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:52:13.877042 (+ 518us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:52:13.877049 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:52:13.877072 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 02:52:13.877284 (+ 212us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:52:13.877295 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:52:13.878423 (+ 1128us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:52:13.878429 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:52:13.879872 (+ 1443us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:52:13.879883 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:52:13.880996 (+ 1113us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:52:13.881000 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:52:13.881137 (+ 137us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:52:13.881220 (+ 83us) client_negotiation.cc:769] Sending connection context
0811 02:52:13.881337 (+ 117us) client_negotiation.cc:240] Negotiation successful
0811 02:52:13.882188 (+ 851us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":897,"mutex_wait_us":742,"thread_start_us":111,"threads_started":1}
I20260811 02:52:13.883206 18290 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:52:13.875414 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50366 (local address 127.12.138.194:34371)
0811 02:52:13.876800 (+ 1386us) server_negotiation.cc:206] Beginning negotiation
0811 02:52:13.876814 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0811 02:52:13.876837 (+ 23us) server_negotiation.cc:407] Connection header received
0811 02:52:13.876916 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:52:13.876921 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:52:13.877180 (+ 259us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:52:13.877340 (+ 160us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 02:52:13.877631 (+ 291us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:52:13.878809 (+ 1178us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:52:13.880317 (+ 1508us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:52:13.880993 (+ 676us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:52:13.882762 (+ 1769us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:52:13.882916 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0811 02:52:13.882964 (+ 48us) server_negotiation.cc:299] Negotiation successful
0811 02:52:13.883054 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1279,"thread_start_us":128,"threads_started":1}
I20260811 02:52:13.885046 17879 tablet_service.cc:1467] Processing CreateTablet for tablet 7e2c38b98efa4873b3164d1c14c42698 (DEFAULT_TABLE table=test-table [id=c78c29fabbf2408cb17a6b299455beb2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 02:52:13.885016 18200 tablet_service.cc:1467] Processing CreateTablet for tablet 7e2c38b98efa4873b3164d1c14c42698 (DEFAULT_TABLE table=test-table [id=c78c29fabbf2408cb17a6b299455beb2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 02:52:13.886125 17879 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7e2c38b98efa4873b3164d1c14c42698. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 02:52:13.886143 18200 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7e2c38b98efa4873b3164d1c14c42698. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 02:52:13.885921 18047 tablet_service.cc:1467] Processing CreateTablet for tablet 7e2c38b98efa4873b3164d1c14c42698 (DEFAULT_TABLE table=test-table [id=c78c29fabbf2408cb17a6b299455beb2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 02:52:13.886879 18047 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7e2c38b98efa4873b3164d1c14c42698. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 02:52:13.893261 18295 tablet_bootstrap.cc:492] T 7e2c38b98efa4873b3164d1c14c42698 P 0d07fa6d87d04eb9b0af513a2a53f8f5: Bootstrap starting.
I20260811 02:52:13.894233 18297 tablet_bootstrap.cc:492] T 7e2c38b98efa4873b3164d1c14c42698 P 4e614b3483fb4f86aab95f6d9846a5aa: Bootstrap starting.
I20260811 02:52:13.894691 18296 tablet_bootstrap.cc:492] T 7e2c38b98efa4873b3164d1c14c42698 P e06050d162db48209c6e3739dd9db64e: Bootstrap starting.
I20260811 02:52:13.895613 18295 tablet_bootstrap.cc:654] T 7e2c38b98efa4873b3164d1c14c42698 P 0d07fa6d87d04eb9b0af513a2a53f8f5: Neither blocks nor log segments found. Creating new log.
I20260811 02:52:13.896376 18295 log.cc:824] T 7e2c38b98efa4873b3164d1c14c42698 P 0d07fa6d87d04eb9b0af513a2a53f8f5: Log is configured to *not* fsync() on all Append() calls
I20260811 02:52:13.896607 18297 tablet_bootstrap.cc:654] T 7e2c38b98efa4873b3164d1c14c42698 P 4e614b3483fb4f86aab95f6d9846a5aa: Neither blocks nor log segments found. Creating new log.
I20260811 02:52:13.896642 18296 tablet_bootstrap.cc:654] T 7e2c38b98efa4873b3164d1c14c42698 P e06050d162db48209c6e3739dd9db64e: Neither blocks nor log segments found. Creating new log.
I20260811 02:52:13.897910 18297 log.cc:824] T 7e2c38b98efa4873b3164d1c14c42698 P 4e614b3483fb4f86aab95f6d9846a5aa: Log is configured to *not* fsync() on all Append() calls
I20260811 02:52:13.897900 18296 log.cc:824] T 7e2c38b98efa4873b3164d1c14c42698 P e06050d162db48209c6e3739dd9db64e: Log is configured to *not* fsync() on all Append() calls
I20260811 02:52:13.899108 18295 tablet_bootstrap.cc:492] T 7e2c38b98efa4873b3164d1c14c42698 P 0d07fa6d87d04eb9b0af513a2a53f8f5: No bootstrap required, opened a new log
I20260811 02:52:13.899318 18295 ts_tablet_manager.cc:1397] T 7e2c38b98efa4873b3164d1c14c42698 P 0d07fa6d87d04eb9b0af513a2a53f8f5: Time spent bootstrapping tablet: real 0.006s user 0.003s sys 0.001s
I20260811 02:52:13.899853 18296 tablet_bootstrap.cc:492] T 7e2c38b98efa4873b3164d1c14c42698 P e06050d162db48209c6e3739dd9db64e: No bootstrap required, opened a new log
I20260811 02:52:13.900041 18296 ts_tablet_manager.cc:1397] T 7e2c38b98efa4873b3164d1c14c42698 P e06050d162db48209c6e3739dd9db64e: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.004s
I20260811 02:52:13.900054 18297 tablet_bootstrap.cc:492] T 7e2c38b98efa4873b3164d1c14c42698 P 4e614b3483fb4f86aab95f6d9846a5aa: No bootstrap required, opened a new log
I20260811 02:52:13.900218 18297 ts_tablet_manager.cc:1397] T 7e2c38b98efa4873b3164d1c14c42698 P 4e614b3483fb4f86aab95f6d9846a5aa: Time spent bootstrapping tablet: real 0.006s user 0.000s sys 0.004s
I20260811 02:52:13.903085 18296 raft_consensus.cc:348] T 7e2c38b98efa4873b3164d1c14c42698 P e06050d162db48209c6e3739dd9db64e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e06050d162db48209c6e3739dd9db64e" member_type: VOTER last_known_addr { host: "127.12.138.194" port: 34371 } } peers { permanent_uuid: "0d07fa6d87d04eb9b0af513a2a53f8f5" member_type: VOTER last_known_addr { host: "127.12.138.193" port: 32807 } } peers { permanent_uuid: "4e614b3483fb4f86aab95f6d9846a5aa" member_type: VOTER last_known_addr { host: "127.12.138.195" port: 35605 } }
I20260811 02:52:13.903434 18296 raft_consensus.cc:374] T 7e2c38b98efa4873b3164d1c14c42698 P e06050d162db48209c6e3739dd9db64e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 02:52:13.903513 18296 raft_consensus.cc:729] T 7e2c38b98efa4873b3164d1c14c42698 P e06050d162db48209c6e3739dd9db64e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e06050d162db48209c6e3739dd9db64e, State: Initialized, Role: FOLLOWER
I20260811 02:52:13.903877 18295 raft_consensus.cc:348] T 7e2c38b98efa4873b3164d1c14c42698 P 0d07fa6d87d04eb9b0af513a2a53f8f5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e06050d162db48209c6e3739dd9db64e" member_type: VOTER last_known_addr { host: "127.12.138.194" port: 34371 } } peers { permanent_uuid: "0d07fa6d87d04eb9b0af513a2a53f8f5" member_type: VOTER last_known_addr { host: "127.12.138.193" port: 32807 } } peers { permanent_uuid: "4e614b3483fb4f86aab95f6d9846a5aa" member_type: VOTER last_known_addr { host: "127.12.138.195" port: 35605 } }
I20260811 02:52:13.904083 18296 consensus_queue.cc:260] T 7e2c38b98efa4873b3164d1c14c42698 P e06050d162db48209c6e3739dd9db64e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e06050d162db48209c6e3739dd9db64e" member_type: VOTER last_known_addr { host: "127.12.138.194" port: 34371 } } peers { permanent_uuid: "0d07fa6d87d04eb9b0af513a2a53f8f5" member_type: VOTER last_known_addr { host: "127.12.138.193" port: 32807 } } peers { permanent_uuid: "4e614b3483fb4f86aab95f6d9846a5aa" member_type: VOTER last_known_addr { host: "127.12.138.195" port: 35605 } }
I20260811 02:52:13.904162 18295 raft_consensus.cc:374] T 7e2c38b98efa4873b3164d1c14c42698 P 0d07fa6d87d04eb9b0af513a2a53f8f5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 02:52:13.904228 18295 raft_consensus.cc:729] T 7e2c38b98efa4873b3164d1c14c42698 P 0d07fa6d87d04eb9b0af513a2a53f8f5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0d07fa6d87d04eb9b0af513a2a53f8f5, State: Initialized, Role: FOLLOWER
I20260811 02:52:13.904228 18297 raft_consensus.cc:348] T 7e2c38b98efa4873b3164d1c14c42698 P 4e614b3483fb4f86aab95f6d9846a5aa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e06050d162db48209c6e3739dd9db64e" member_type: VOTER last_known_addr { host: "127.12.138.194" port: 34371 } } peers { permanent_uuid: "0d07fa6d87d04eb9b0af513a2a53f8f5" member_type: VOTER last_known_addr { host: "127.12.138.193" port: 32807 } } peers { permanent_uuid: "4e614b3483fb4f86aab95f6d9846a5aa" member_type: VOTER last_known_addr { host: "127.12.138.195" port: 35605 } }
I20260811 02:52:13.904512 18297 raft_consensus.cc:374] T 7e2c38b98efa4873b3164d1c14c42698 P 4e614b3483fb4f86aab95f6d9846a5aa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 02:52:13.904585 18297 raft_consensus.cc:729] T 7e2c38b98efa4873b3164d1c14c42698 P 4e614b3483fb4f86aab95f6d9846a5aa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4e614b3483fb4f86aab95f6d9846a5aa, State: Initialized, Role: FOLLOWER
I20260811 02:52:13.904744 18295 consensus_queue.cc:260] T 7e2c38b98efa4873b3164d1c14c42698 P 0d07fa6d87d04eb9b0af513a2a53f8f5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e06050d162db48209c6e3739dd9db64e" member_type: VOTER last_known_addr { host: "127.12.138.194" port: 34371 } } peers { permanent_uuid: "0d07fa6d87d04eb9b0af513a2a53f8f5" member_type: VOTER last_known_addr { host: "127.12.138.193" port: 32807 } } peers { permanent_uuid: "4e614b3483fb4f86aab95f6d9846a5aa" member_type: VOTER last_known_addr { host: "127.12.138.195" port: 35605 } }
I20260811 02:52:13.905040 18297 consensus_queue.cc:260] T 7e2c38b98efa4873b3164d1c14c42698 P 4e614b3483fb4f86aab95f6d9846a5aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e06050d162db48209c6e3739dd9db64e" member_type: VOTER last_known_addr { host: "127.12.138.194" port: 34371 } } peers { permanent_uuid: "0d07fa6d87d04eb9b0af513a2a53f8f5" member_type: VOTER last_known_addr { host: "127.12.138.193" port: 32807 } } peers { permanent_uuid: "4e614b3483fb4f86aab95f6d9846a5aa" member_type: VOTER last_known_addr { host: "127.12.138.195" port: 35605 } }
I20260811 02:52:13.905223 18296 ts_tablet_manager.cc:1428] T 7e2c38b98efa4873b3164d1c14c42698 P e06050d162db48209c6e3739dd9db64e: Time spent starting tablet: real 0.005s user 0.005s sys 0.001s
I20260811 02:52:13.905491 17961 heartbeater.cc:503] Master 127.12.138.254:45359 was elected leader, sending a full tablet report...
I20260811 02:52:13.905598 18121 heartbeater.cc:503] Master 127.12.138.254:45359 was elected leader, sending a full tablet report...
I20260811 02:52:13.905748 18295 ts_tablet_manager.cc:1428] T 7e2c38b98efa4873b3164d1c14c42698 P 0d07fa6d87d04eb9b0af513a2a53f8f5: Time spent starting tablet: real 0.006s user 0.004s sys 0.002s
I20260811 02:52:13.905977 18297 ts_tablet_manager.cc:1428] T 7e2c38b98efa4873b3164d1c14c42698 P 4e614b3483fb4f86aab95f6d9846a5aa: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260811 02:52:13.906312 18275 heartbeater.cc:503] Master 127.12.138.254:45359 was elected leader, sending a full tablet report...
I20260811 02:52:13.943284 18302 raft_consensus.cc:482] T 7e2c38b98efa4873b3164d1c14c42698 P 0d07fa6d87d04eb9b0af513a2a53f8f5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 02:52:13.943507 18302 raft_consensus.cc:504] T 7e2c38b98efa4873b3164d1c14c42698 P 0d07fa6d87d04eb9b0af513a2a53f8f5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e06050d162db48209c6e3739dd9db64e" member_type: VOTER last_known_addr { host: "127.12.138.194" port: 34371 } } peers { permanent_uuid: "0d07fa6d87d04eb9b0af513a2a53f8f5" member_type: VOTER last_known_addr { host: "127.12.138.193" port: 32807 } } peers { permanent_uuid: "4e614b3483fb4f86aab95f6d9846a5aa" member_type: VOTER last_known_addr { host: "127.12.138.195" port: 35605 } }
I20260811 02:52:13.944759 18302 leader_election.cc:288] T 7e2c38b98efa4873b3164d1c14c42698 P 0d07fa6d87d04eb9b0af513a2a53f8f5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e06050d162db48209c6e3739dd9db64e (127.12.138.194:34371), 4e614b3483fb4f86aab95f6d9846a5aa (127.12.138.195:35605)
I20260811 02:52:13.949146 18305 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:52:13.945541 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.138.195:35605 (local address 127.12.138.193:57271)
0811 02:52:13.945879 (+ 338us) negotiation.cc:107] Waiting for socket to connect
0811 02:52:13.945891 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 02:52:13.945987 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:52:13.946226 (+ 239us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:52:13.946228 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:52:13.946241 (+ 13us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 02:52:13.946380 (+ 139us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:52:13.946386 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:52:13.947133 (+ 747us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:52:13.947135 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:52:13.948012 (+ 877us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:52:13.948022 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:52:13.948672 (+ 650us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:52:13.948676 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:52:13.948782 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:52:13.948821 (+ 39us) client_negotiation.cc:769] Sending connection context
0811 02:52:13.948893 (+ 72us) client_negotiation.cc:240] Negotiation successful
0811 02:52:13.948974 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":273,"mutex_wait_us":36,"thread_start_us":108,"threads_started":1}
I20260811 02:52:13.949172 18294 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:52:13.945717 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.138.193:57271 (local address 127.12.138.195:35605)
0811 02:52:13.945879 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0811 02:52:13.945884 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 02:52:13.945980 (+ 96us) server_negotiation.cc:407] Connection header received
0811 02:52:13.946062 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:52:13.946065 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:52:13.946113 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:52:13.946189 (+ 76us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 02:52:13.946493 (+ 304us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:52:13.947024 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:52:13.948137 (+ 1113us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:52:13.948556 (+ 419us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:52:13.948649 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:52:13.948690 (+ 41us) server_negotiation.cc:1035] Waiting for connection context
0811 02:52:13.948929 (+ 239us) server_negotiation.cc:299] Negotiation successful
0811 02:52:13.949001 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260811 02:52:13.949375 18290 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:52:13.945238 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.138.193:40107 (local address 127.12.138.194:34371)
0811 02:52:13.945384 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0811 02:52:13.945388 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 02:52:13.945513 (+ 125us) server_negotiation.cc:407] Connection header received
0811 02:52:13.945715 (+ 202us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:52:13.945719 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:52:13.945774 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:52:13.945848 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 02:52:13.946356 (+ 508us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:52:13.946944 (+ 588us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:52:13.948173 (+ 1229us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:52:13.948730 (+ 557us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:52:13.948852 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:52:13.948912 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0811 02:52:13.949116 (+ 204us) server_negotiation.cc:299] Negotiation successful
0811 02:52:13.949230 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260811 02:52:13.949536 18304 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:52:13.945046 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.138.194:34371 (local address 127.12.138.193:40107)
0811 02:52:13.945390 (+ 344us) negotiation.cc:107] Waiting for socket to connect
0811 02:52:13.945405 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 02:52:13.945526 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:52:13.946051 (+ 525us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:52:13.946054 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:52:13.946074 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 02:52:13.946229 (+ 155us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:52:13.946236 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:52:13.947064 (+ 828us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:52:13.947069 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:52:13.948025 (+ 956us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:52:13.948034 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:52:13.948856 (+ 822us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:52:13.948859 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:52:13.948983 (+ 124us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:52:13.949006 (+ 23us) client_negotiation.cc:769] Sending connection context
0811 02:52:13.949083 (+ 77us) client_negotiation.cc:240] Negotiation successful
0811 02:52:13.949177 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":269,"mutex_wait_us":53,"thread_start_us":171,"threads_started":1}
I20260811 02:52:13.949930 18223 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7e2c38b98efa4873b3164d1c14c42698" candidate_uuid: "0d07fa6d87d04eb9b0af513a2a53f8f5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4e614b3483fb4f86aab95f6d9846a5aa" is_pre_election: true
I20260811 02:52:13.950279 18223 raft_consensus.cc:2454] T 7e2c38b98efa4873b3164d1c14c42698 P 4e614b3483fb4f86aab95f6d9846a5aa [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0d07fa6d87d04eb9b0af513a2a53f8f5 in term 0.
I20260811 02:52:13.950788 18069 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7e2c38b98efa4873b3164d1c14c42698" candidate_uuid: "0d07fa6d87d04eb9b0af513a2a53f8f5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e06050d162db48209c6e3739dd9db64e" is_pre_election: true
I20260811 02:52:13.950819 17825 leader_election.cc:302] T 7e2c38b98efa4873b3164d1c14c42698 P 0d07fa6d87d04eb9b0af513a2a53f8f5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0d07fa6d87d04eb9b0af513a2a53f8f5, 4e614b3483fb4f86aab95f6d9846a5aa; no voters:
I20260811 02:52:13.951072 18069 raft_consensus.cc:2454] T 7e2c38b98efa4873b3164d1c14c42698 P e06050d162db48209c6e3739dd9db64e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0d07fa6d87d04eb9b0af513a2a53f8f5 in term 0.
I20260811 02:52:13.951097 18302 raft_consensus.cc:2781] T 7e2c38b98efa4873b3164d1c14c42698 P 0d07fa6d87d04eb9b0af513a2a53f8f5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 02:52:13.951182 18302 raft_consensus.cc:482] T 7e2c38b98efa4873b3164d1c14c42698 P 0d07fa6d87d04eb9b0af513a2a53f8f5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 02:52:13.951234 18302 raft_consensus.cc:3037] T 7e2c38b98efa4873b3164d1c14c42698 P 0d07fa6d87d04eb9b0af513a2a53f8f5 [term 0 FOLLOWER]: Advancing to term 1
I20260811 02:52:13.952526 18302 raft_consensus.cc:504] T 7e2c38b98efa4873b3164d1c14c42698 P 0d07fa6d87d04eb9b0af513a2a53f8f5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e06050d162db48209c6e3739dd9db64e" member_type: VOTER last_known_addr { host: "127.12.138.194" port: 34371 } } peers { permanent_uuid: "0d07fa6d87d04eb9b0af513a2a53f8f5" member_type: VOTER last_known_addr { host: "127.12.138.193" port: 32807 } } peers { permanent_uuid: "4e614b3483fb4f86aab95f6d9846a5aa" member_type: VOTER last_known_addr { host: "127.12.138.195" port: 35605 } }
I20260811 02:52:13.952891 18302 leader_election.cc:288] T 7e2c38b98efa4873b3164d1c14c42698 P 0d07fa6d87d04eb9b0af513a2a53f8f5 [CANDIDATE]: Term 1 election: Requested vote from peers e06050d162db48209c6e3739dd9db64e (127.12.138.194:34371), 4e614b3483fb4f86aab95f6d9846a5aa (127.12.138.195:35605)
I20260811 02:52:13.953233 18069 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7e2c38b98efa4873b3164d1c14c42698" candidate_uuid: "0d07fa6d87d04eb9b0af513a2a53f8f5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e06050d162db48209c6e3739dd9db64e"
I20260811 02:52:13.953293 18223 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7e2c38b98efa4873b3164d1c14c42698" candidate_uuid: "0d07fa6d87d04eb9b0af513a2a53f8f5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4e614b3483fb4f86aab95f6d9846a5aa"
I20260811 02:52:13.953400 18223 raft_consensus.cc:3037] T 7e2c38b98efa4873b3164d1c14c42698 P 4e614b3483fb4f86aab95f6d9846a5aa [term 0 FOLLOWER]: Advancing to term 1
I20260811 02:52:13.953387 18069 raft_consensus.cc:3037] T 7e2c38b98efa4873b3164d1c14c42698 P e06050d162db48209c6e3739dd9db64e [term 0 FOLLOWER]: Advancing to term 1
I20260811 02:52:13.954701 18069 raft_consensus.cc:2454] T 7e2c38b98efa4873b3164d1c14c42698 P e06050d162db48209c6e3739dd9db64e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0d07fa6d87d04eb9b0af513a2a53f8f5 in term 1.
I20260811 02:52:13.954695 18223 raft_consensus.cc:2454] T 7e2c38b98efa4873b3164d1c14c42698 P 4e614b3483fb4f86aab95f6d9846a5aa [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0d07fa6d87d04eb9b0af513a2a53f8f5 in term 1.
I20260811 02:52:13.954998 17825 leader_election.cc:302] T 7e2c38b98efa4873b3164d1c14c42698 P 0d07fa6d87d04eb9b0af513a2a53f8f5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0d07fa6d87d04eb9b0af513a2a53f8f5, e06050d162db48209c6e3739dd9db64e; no voters:
I20260811 02:52:13.955193 18302 raft_consensus.cc:2781] T 7e2c38b98efa4873b3164d1c14c42698 P 0d07fa6d87d04eb9b0af513a2a53f8f5 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 02:52:13.955435 18302 raft_consensus.cc:686] T 7e2c38b98efa4873b3164d1c14c42698 P 0d07fa6d87d04eb9b0af513a2a53f8f5 [term 1 LEADER]: Becoming Leader. State: Replica: 0d07fa6d87d04eb9b0af513a2a53f8f5, State: Running, Role: LEADER
I20260811 02:52:13.955750 18302 consensus_queue.cc:237] T 7e2c38b98efa4873b3164d1c14c42698 P 0d07fa6d87d04eb9b0af513a2a53f8f5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e06050d162db48209c6e3739dd9db64e" member_type: VOTER last_known_addr { host: "127.12.138.194" port: 34371 } } peers { permanent_uuid: "0d07fa6d87d04eb9b0af513a2a53f8f5" member_type: VOTER last_known_addr { host: "127.12.138.193" port: 32807 } } peers { permanent_uuid: "4e614b3483fb4f86aab95f6d9846a5aa" member_type: VOTER last_known_addr { host: "127.12.138.195" port: 35605 } }
I20260811 02:52:13.959007 17732 catalog_manager.cc:5495] T 7e2c38b98efa4873b3164d1c14c42698 P 0d07fa6d87d04eb9b0af513a2a53f8f5 reported cstate change: term changed from 0 to 1, leader changed from <none> to 0d07fa6d87d04eb9b0af513a2a53f8f5 (127.12.138.193). New cstate: current_term: 1 leader_uuid: "0d07fa6d87d04eb9b0af513a2a53f8f5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e06050d162db48209c6e3739dd9db64e" member_type: VOTER last_known_addr { host: "127.12.138.194" port: 34371 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0d07fa6d87d04eb9b0af513a2a53f8f5" member_type: VOTER last_known_addr { host: "127.12.138.193" port: 32807 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4e614b3483fb4f86aab95f6d9846a5aa" member_type: VOTER last_known_addr { host: "127.12.138.195" port: 35605 } health_report { overall_health: UNKNOWN } } }
I20260811 02:52:13.982715 18293 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:52:13.976095 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40078 (local address 127.12.138.193:32807)
0811 02:52:13.976236 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0811 02:52:13.976240 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 02:52:13.976255 (+ 15us) server_negotiation.cc:407] Connection header received
0811 02:52:13.976373 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:52:13.976376 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:52:13.976429 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:52:13.976518 (+ 89us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 02:52:13.976749 (+ 231us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:52:13.977382 (+ 633us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:52:13.978677 (+ 1295us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:52:13.980974 (+ 2297us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:52:13.981087 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:52:13.981638 (+ 551us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 02:52:13.982273 (+ 635us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 02:52:13.982416 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0811 02:52:13.982461 (+ 45us) server_negotiation.cc:299] Negotiation successful
0811 02:52:13.982539 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260811 02:52:13.989684 18069 raft_consensus.cc:1261] T 7e2c38b98efa4873b3164d1c14c42698 P e06050d162db48209c6e3739dd9db64e [term 1 FOLLOWER]: Refusing update from remote peer 0d07fa6d87d04eb9b0af513a2a53f8f5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 02:52:13.989779 18223 raft_consensus.cc:1261] T 7e2c38b98efa4873b3164d1c14c42698 P 4e614b3483fb4f86aab95f6d9846a5aa [term 1 FOLLOWER]: Refusing update from remote peer 0d07fa6d87d04eb9b0af513a2a53f8f5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 02:52:13.990465 18302 consensus_queue.cc:1035] T 7e2c38b98efa4873b3164d1c14c42698 P 0d07fa6d87d04eb9b0af513a2a53f8f5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e06050d162db48209c6e3739dd9db64e" member_type: VOTER last_known_addr { host: "127.12.138.194" port: 34371 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 02:52:13.990828 18306 consensus_queue.cc:1035] T 7e2c38b98efa4873b3164d1c14c42698 P 0d07fa6d87d04eb9b0af513a2a53f8f5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e614b3483fb4f86aab95f6d9846a5aa" member_type: VOTER last_known_addr { host: "127.12.138.195" port: 35605 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 02:52:13.999651 18309 mvcc.cc:205] Tried to move back new op lower bound from 7317162942413373440 to 7317162942285418496. Current Snapshot: MvccSnapshot[applied={T|T < 7317162942413373440}]
I20260811 02:52:14.001315 18310 mvcc.cc:205] Tried to move back new op lower bound from 7317162942413373440 to 7317162942285418496. Current Snapshot: MvccSnapshot[applied={T|T < 7317162942413373440}]
I20260811 02:52:14.003577 18311 mvcc.cc:205] Tried to move back new op lower bound from 7317162942413373440 to 7317162942285418496. Current Snapshot: MvccSnapshot[applied={T|T < 7317162942413373440 or (T in {7317162942413373440})}]
I20260811 02:52:14.011312 17731 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57834:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 02:52:14.011514 17731 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57834:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 02:52:14.014669 17731 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 30098621f7d7498b8e58014c89f48ed3: Sending DeleteTablet for 3 replicas of tablet 7e2c38b98efa4873b3164d1c14c42698
I20260811 02:52:14.015691 17879 tablet_service.cc:1514] Processing DeleteTablet for tablet 7e2c38b98efa4873b3164d1c14c42698 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 02:52:14 UTC) from {username='kudu', principal='kudu/127.12.138.254@KRBTEST.COM'} at 127.0.0.1:40070
I20260811 02:52:14.015866 18047 tablet_service.cc:1514] Processing DeleteTablet for tablet 7e2c38b98efa4873b3164d1c14c42698 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 02:52:14 UTC) from {username='kudu', principal='kudu/127.12.138.254@KRBTEST.COM'} at 127.0.0.1:50366
I20260811 02:52:14.016407 18200 tablet_service.cc:1514] Processing DeleteTablet for tablet 7e2c38b98efa4873b3164d1c14c42698 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 02:52:14 UTC) from {username='kudu', principal='kudu/127.12.138.254@KRBTEST.COM'} at 127.0.0.1:57852
I20260811 02:52:14.016599 12843 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDawEwS/build/debug/bin/kudu with pid 17804
I20260811 02:52:14.016636 18321 tablet_replica.cc:330] T 7e2c38b98efa4873b3164d1c14c42698 P e06050d162db48209c6e3739dd9db64e: stopping tablet replica
I20260811 02:52:14.021296 18321 raft_consensus.cc:2229] T 7e2c38b98efa4873b3164d1c14c42698 P e06050d162db48209c6e3739dd9db64e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 02:52:14.021677 18321 raft_consensus.cc:2258] T 7e2c38b98efa4873b3164d1c14c42698 P e06050d162db48209c6e3739dd9db64e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 02:52:14.022632 18322 tablet_replica.cc:330] T 7e2c38b98efa4873b3164d1c14c42698 P 4e614b3483fb4f86aab95f6d9846a5aa: stopping tablet replica
I20260811 02:52:14.022984 18322 raft_consensus.cc:2229] T 7e2c38b98efa4873b3164d1c14c42698 P 4e614b3483fb4f86aab95f6d9846a5aa [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 02:52:14.023290 18322 raft_consensus.cc:2258] T 7e2c38b98efa4873b3164d1c14c42698 P 4e614b3483fb4f86aab95f6d9846a5aa [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 02:52:14.023854 18321 ts_tablet_manager.cc:1905] T 7e2c38b98efa4873b3164d1c14c42698 P e06050d162db48209c6e3739dd9db64e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 02:52:14.024883 18322 ts_tablet_manager.cc:1905] T 7e2c38b98efa4873b3164d1c14c42698 P 4e614b3483fb4f86aab95f6d9846a5aa: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 02:52:14.026645 18321 ts_tablet_manager.cc:1918] T 7e2c38b98efa4873b3164d1c14c42698 P e06050d162db48209c6e3739dd9db64e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 02:52:14.026786 18321 log.cc:1196] T 7e2c38b98efa4873b3164d1c14c42698 P e06050d162db48209c6e3739dd9db64e: Deleting WAL directory at /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786416707705494-12843-0/minicluster-data/ts-1/wal/wals/7e2c38b98efa4873b3164d1c14c42698
I20260811 02:52:14.027212 18321 ts_tablet_manager.cc:1939] T 7e2c38b98efa4873b3164d1c14c42698 P e06050d162db48209c6e3739dd9db64e: Deleting consensus metadata
I20260811 02:52:14.028504 18322 ts_tablet_manager.cc:1918] T 7e2c38b98efa4873b3164d1c14c42698 P 4e614b3483fb4f86aab95f6d9846a5aa: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 02:52:14.028611 18322 log.cc:1196] T 7e2c38b98efa4873b3164d1c14c42698 P 4e614b3483fb4f86aab95f6d9846a5aa: Deleting WAL directory at /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786416707705494-12843-0/minicluster-data/ts-2/wal/wals/7e2c38b98efa4873b3164d1c14c42698
I20260811 02:52:14.028792 17717 catalog_manager.cc:4841] TS e06050d162db48209c6e3739dd9db64e (127.12.138.194:34371): tablet 7e2c38b98efa4873b3164d1c14c42698 (table test-table [id=c78c29fabbf2408cb17a6b299455beb2]) successfully deleted
I20260811 02:52:14.028975 18322 ts_tablet_manager.cc:1939] T 7e2c38b98efa4873b3164d1c14c42698 P 4e614b3483fb4f86aab95f6d9846a5aa: Deleting consensus metadata
W20260811 02:52:14.029753 17718 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.12.138.193:32807 (error 108)
I20260811 02:52:14.029795 12843 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDawEwS/build/debug/bin/kudu with pid 17967
I20260811 02:52:14.032863 17717 catalog_manager.cc:4841] TS 4e614b3483fb4f86aab95f6d9846a5aa (127.12.138.195:35605): tablet 7e2c38b98efa4873b3164d1c14c42698 (table test-table [id=c78c29fabbf2408cb17a6b299455beb2]) successfully deleted
I20260811 02:52:14.034396 18291 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:52:14.034100 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.138.193:32807 (local address 127.0.0.1:40084)
0811 02:52:14.034246 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0811 02:52:14.034321 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.138.193:32807: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
W20260811 02:52:14.034557 17718 catalog_manager.cc:4584] TS 0d07fa6d87d04eb9b0af513a2a53f8f5 (127.12.138.193:32807): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 7e2c38b98efa4873b3164d1c14c42698: Network error: Client connection negotiation failed: client connection to 127.12.138.193:32807: connect: Connection refused (error 111)
I20260811 02:52:14.040035 12843 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDawEwS/build/debug/bin/kudu with pid 18126
I20260811 02:52:14.052654 12843 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDawEwS/build/debug/bin/kudu with pid 17691
2026-08-11T02:52:14Z chronyd exiting
[ OK ] SecurityITest.TestDisableWebUI (2513 ms)
[ RUN ] SecurityITest.TestDisableAuthenticationEncryption
2026-08-11T02:52:14Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T02:52:14Z Disabled control of system clock
I20260811 02:52:14.089622 12843 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDawEwS/build/debug/bin/kudu
/tmp/dist-test-taskDawEwS/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786416707705494-12843-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786416707705494-12843-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786416707705494-12843-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786416707705494-12843-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.138.254:37155
--webserver_interface=127.12.138.254
--webserver_port=0
--builtin_ntp_servers=127.12.138.212:40813
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.12.138.254:37155
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260811 02:52:14.206682 18328 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 02:52:14.206990 18328 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 02:52:14.207029 18328 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 02:52:14.211112 18328 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 02:52:14.211176 18328 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 02:52:14.211192 18328 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 02:52:14.211210 18328 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 02:52:14.211225 18328 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 02:52:14.216135 18328 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.138.212:40813
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786416707705494-12843-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786416707705494-12843-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.12.138.254:37155
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.138.254:37155
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786416707705494-12843-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.12.138.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18328
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786416707705494-12843-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 02:09:49 UTC on ccycloud.quasar-fsnqqc.root.comops.site
build id 4579570
I20260811 02:52:14.217254 18328 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 02:52:14.218634 18328 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 02:52:14.227232 18336 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:52:14.227270 18339 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:52:14.227232 18335 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:52:14.227425 18328 server_base.cc:1034] running on GCE node
I20260811 02:52:14.228016 18328 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 02:52:14.229132 18328 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 02:52:14.230376 18328 hybrid_clock.cc:648] HybridClock initialized: now 1786416734230362 us; error 26 us; skew 500 ppm
I20260811 02:52:14.232343 18328 webserver.cc:460] Webserver started at http://127.12.138.254:45343/ using document root <none> and password file <none>
I20260811 02:52:14.232939 18328 fs_manager.cc:359] Metadata directory not provided
I20260811 02:52:14.233033 18328 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 02:52:14.233323 18328 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 02:52:14.235440 18328 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786416707705494-12843-0/minicluster-data/master-0/data/instance:
uuid: "f69489d1baf24b26b6785af7aa995e40"
format_stamp: "Formatted at 2026-08-11 02:52:14 on dist-test-slave-9gcw"
server_key: "5c6cb2b9cb0315fbcbb8179fd700111b"
server_key_iv: "2e14efa26292aa5c43a18c537e2d2051"
server_key_version: "encryptionkey@0"
I20260811 02:52:14.236033 18328 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786416707705494-12843-0/minicluster-data/master-0/wal/instance:
uuid: "f69489d1baf24b26b6785af7aa995e40"
format_stamp: "Formatted at 2026-08-11 02:52:14 on dist-test-slave-9gcw"
server_key: "5c6cb2b9cb0315fbcbb8179fd700111b"
server_key_iv: "2e14efa26292aa5c43a18c537e2d2051"
server_key_version: "encryptionkey@0"
I20260811 02:52:14.239712 18328 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.001s
I20260811 02:52:14.242237 18345 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 02:52:14.243480 18328 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260811 02:52:14.243623 18328 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786416707705494-12843-0/minicluster-data/master-0/data,/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786416707705494-12843-0/minicluster-data/master-0/wal
uuid: "f69489d1baf24b26b6785af7aa995e40"
format_stamp: "Formatted at 2026-08-11 02:52:14 on dist-test-slave-9gcw"
server_key: "5c6cb2b9cb0315fbcbb8179fd700111b"
server_key_iv: "2e14efa26292aa5c43a18c537e2d2051"
server_key_version: "encryptionkey@0"
I20260811 02:52:14.243741 18328 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786416707705494-12843-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786416707705494-12843-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786416707705494-12843-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 02:52:14.248313 18328 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 02:52:14.248934 18328 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 02:52:14.249127 18328 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 02:52:14.256160 18328 rpc_server.cc:307] RPC server started. Bound to: 127.12.138.254:37155
I20260811 02:52:14.256170 18408 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.138.254:37155 every 8 connection(s)
I20260811 02:52:14.257114 18328 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786416707705494-12843-0/minicluster-data/master-0/data/info.pb
I20260811 02:52:14.260397 18409 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 02:52:14.265923 12843 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDawEwS/build/debug/bin/kudu as pid 18328
I20260811 02:52:14.266072 12843 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786416707705494-12843-0/minicluster-data/master-0/wal/instance
I20260811 02:52:14.266376 12843 external_mini_cluster.cc:1428] Setting key 76469893e1293fd1e1923db5fd2a3b31
I20260811 02:52:14.266541 18409 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f69489d1baf24b26b6785af7aa995e40: Bootstrap starting.
I20260811 02:52:14.269310 18409 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f69489d1baf24b26b6785af7aa995e40: Neither blocks nor log segments found. Creating new log.
I20260811 02:52:14.270090 18409 log.cc:824] T 00000000000000000000000000000000 P f69489d1baf24b26b6785af7aa995e40: Log is configured to *not* fsync() on all Append() calls
I20260811 02:52:14.272289 18409 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f69489d1baf24b26b6785af7aa995e40: No bootstrap required, opened a new log
I20260811 02:52:14.272616 18413 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:52:14.267901 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53536 (local address 127.12.138.254:37155)
0811 02:52:14.268614 (+ 713us) server_negotiation.cc:206] Beginning negotiation
0811 02:52:14.268631 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0811 02:52:14.268667 (+ 36us) server_negotiation.cc:407] Connection header received
0811 02:52:14.269455 (+ 788us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:52:14.269480 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:52:14.269864 (+ 384us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:52:14.270222 (+ 358us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:52:14.271146 (+ 924us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:52:14.271151 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:52:14.271163 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 02:52:14.271174 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:52:14.271239 (+ 65us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 02:52:14.271256 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:52:14.271362 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0811 02:52:14.271509 (+ 147us) server_negotiation.cc:299] Negotiation successful
0811 02:52:14.271855 (+ 346us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":339,"thread_start_us":161,"threads_started":1}
I20260811 02:52:14.276554 18409 raft_consensus.cc:348] T 00000000000000000000000000000000 P f69489d1baf24b26b6785af7aa995e40 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f69489d1baf24b26b6785af7aa995e40" member_type: VOTER last_known_addr { host: "127.12.138.254" port: 37155 } }
I20260811 02:52:14.276862 18409 raft_consensus.cc:374] T 00000000000000000000000000000000 P f69489d1baf24b26b6785af7aa995e40 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 02:52:14.276944 18409 raft_consensus.cc:729] T 00000000000000000000000000000000 P f69489d1baf24b26b6785af7aa995e40 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f69489d1baf24b26b6785af7aa995e40, State: Initialized, Role: FOLLOWER
I20260811 02:52:14.277424 18409 consensus_queue.cc:260] T 00000000000000000000000000000000 P f69489d1baf24b26b6785af7aa995e40 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f69489d1baf24b26b6785af7aa995e40" member_type: VOTER last_known_addr { host: "127.12.138.254" port: 37155 } }
I20260811 02:52:14.277572 18409 raft_consensus.cc:388] T 00000000000000000000000000000000 P f69489d1baf24b26b6785af7aa995e40 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 02:52:14.277642 18409 raft_consensus.cc:482] T 00000000000000000000000000000000 P f69489d1baf24b26b6785af7aa995e40 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 02:52:14.277765 18409 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f69489d1baf24b26b6785af7aa995e40 [term 0 FOLLOWER]: Advancing to term 1
I20260811 02:52:14.278748 18409 raft_consensus.cc:504] T 00000000000000000000000000000000 P f69489d1baf24b26b6785af7aa995e40 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f69489d1baf24b26b6785af7aa995e40" member_type: VOTER last_known_addr { host: "127.12.138.254" port: 37155 } }
I20260811 02:52:14.279093 18409 leader_election.cc:302] T 00000000000000000000000000000000 P f69489d1baf24b26b6785af7aa995e40 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f69489d1baf24b26b6785af7aa995e40; no voters:
I20260811 02:52:14.279404 18409 leader_election.cc:288] T 00000000000000000000000000000000 P f69489d1baf24b26b6785af7aa995e40 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 02:52:14.279557 18416 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f69489d1baf24b26b6785af7aa995e40 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 02:52:14.279808 18416 raft_consensus.cc:686] T 00000000000000000000000000000000 P f69489d1baf24b26b6785af7aa995e40 [term 1 LEADER]: Becoming Leader. State: Replica: f69489d1baf24b26b6785af7aa995e40, State: Running, Role: LEADER
I20260811 02:52:14.280121 18416 consensus_queue.cc:237] T 00000000000000000000000000000000 P f69489d1baf24b26b6785af7aa995e40 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f69489d1baf24b26b6785af7aa995e40" member_type: VOTER last_known_addr { host: "127.12.138.254" port: 37155 } }
I20260811 02:52:14.280745 18409 sys_catalog.cc:564] T 00000000000000000000000000000000 P f69489d1baf24b26b6785af7aa995e40 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 02:52:14.282120 18418 sys_catalog.cc:455] T 00000000000000000000000000000000 P f69489d1baf24b26b6785af7aa995e40 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f69489d1baf24b26b6785af7aa995e40" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f69489d1baf24b26b6785af7aa995e40" member_type: VOTER last_known_addr { host: "127.12.138.254" port: 37155 } } }
I20260811 02:52:14.282131 18419 sys_catalog.cc:455] T 00000000000000000000000000000000 P f69489d1baf24b26b6785af7aa995e40 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f69489d1baf24b26b6785af7aa995e40. Latest consensus state: current_term: 1 leader_uuid: "f69489d1baf24b26b6785af7aa995e40" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f69489d1baf24b26b6785af7aa995e40" member_type: VOTER last_known_addr { host: "127.12.138.254" port: 37155 } } }
I20260811 02:52:14.282269 18419 sys_catalog.cc:458] T 00000000000000000000000000000000 P f69489d1baf24b26b6785af7aa995e40 [sys.catalog]: This master's current role is: LEADER
I20260811 02:52:14.282254 18418 sys_catalog.cc:458] T 00000000000000000000000000000000 P f69489d1baf24b26b6785af7aa995e40 [sys.catalog]: This master's current role is: LEADER
I20260811 02:52:14.282750 18428 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 02:52:14.285935 18428 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 02:52:14.291599 18428 catalog_manager.cc:1347] Generated new cluster ID: 9c075a56c2f740baacb0998a2f72ffa8
I20260811 02:52:14.291668 18428 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 02:52:14.305655 18428 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 02:52:14.306028 18428 catalog_manager.cc:1504] Loading token signing keys...
I20260811 02:52:14.313130 18428 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f69489d1baf24b26b6785af7aa995e40: Generated new TSK 0
I20260811 02:52:14.313907 18428 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 02:52:14.318562 12843 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDawEwS/build/debug/bin/kudu
/tmp/dist-test-taskDawEwS/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786416707705494-12843-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786416707705494-12843-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786416707705494-12843-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786416707705494-12843-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.138.193:0
--local_ip_for_outbound_sockets=127.12.138.193
--webserver_interface=127.12.138.193
--webserver_port=0
--tserver_master_addrs=127.12.138.254:37155
--builtin_ntp_servers=127.12.138.212:40813
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260811 02:52:14.437546 18439 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 02:52:14.437812 18439 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 02:52:14.437853 18439 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 02:52:14.441859 18439 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 02:52:14.441923 18439 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 02:52:14.441980 18439 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.138.193
I20260811 02:52:14.446843 18439 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.138.212:40813
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786416707705494-12843-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786416707705494-12843-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.138.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786416707705494-12843-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.12.138.193
--webserver_port=0
--tserver_master_addrs=127.12.138.254:37155
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18439
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.138.193
--log_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786416707705494-12843-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 02:09:49 UTC on ccycloud.quasar-fsnqqc.root.comops.site
build id 4579570
I20260811 02:52:14.448103 18439 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 02:52:14.449643 18439 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 02:52:14.460041 18448 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:52:14.460107 18445 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:52:14.460289 18439 server_base.cc:1034] running on GCE node
W20260811 02:52:14.460041 18446 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:52:14.460796 18439 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 02:52:14.461534 18439 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 02:52:14.462757 18439 hybrid_clock.cc:648] HybridClock initialized: now 1786416734462724 us; error 44 us; skew 500 ppm
I20260811 02:52:14.464815 18439 webserver.cc:460] Webserver started at http://127.12.138.193:39195/ using document root <none> and password file <none>
I20260811 02:52:14.465476 18439 fs_manager.cc:359] Metadata directory not provided
I20260811 02:52:14.465576 18439 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 02:52:14.465849 18439 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 02:52:14.467957 18439 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786416707705494-12843-0/minicluster-data/ts-0/data/instance:
uuid: "ea24344f7c15420595cefc09fb128be0"
format_stamp: "Formatted at 2026-08-11 02:52:14 on dist-test-slave-9gcw"
server_key: "b05ad70c09e4200d2479a7380bddacd8"
server_key_iv: "288bdf75e4e67322d3e027f5b3dd030b"
server_key_version: "encryptionkey@0"
I20260811 02:52:14.468555 18439 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786416707705494-12843-0/minicluster-data/ts-0/wal/instance:
uuid: "ea24344f7c15420595cefc09fb128be0"
format_stamp: "Formatted at 2026-08-11 02:52:14 on dist-test-slave-9gcw"
server_key: "b05ad70c09e4200d2479a7380bddacd8"
server_key_iv: "288bdf75e4e67322d3e027f5b3dd030b"
server_key_version: "encryptionkey@0"
I20260811 02:52:14.472352 18439 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260811 02:52:14.475093 18455 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 02:52:14.476439 18439 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260811 02:52:14.476598 18439 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786416707705494-12843-0/minicluster-data/ts-0/data,/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786416707705494-12843-0/minicluster-data/ts-0/wal
uuid: "ea24344f7c15420595cefc09fb128be0"
format_stamp: "Formatted at 2026-08-11 02:52:14 on dist-test-slave-9gcw"
server_key: "b05ad70c09e4200d2479a7380bddacd8"
server_key_iv: "288bdf75e4e67322d3e027f5b3dd030b"
server_key_version: "encryptionkey@0"
I20260811 02:52:14.476730 18439 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786416707705494-12843-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786416707705494-12843-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786416707705494-12843-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 02:52:14.481532 18439 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 02:52:14.482240 18439 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 02:52:14.482427 18439 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 02:52:14.483042 18439 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 02:52:14.483986 18439 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 02:52:14.484059 18439 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 02:52:14.484124 18439 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 02:52:14.484177 18439 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 02:52:14.492880 18439 rpc_server.cc:307] RPC server started. Bound to: 127.12.138.193:32779
I20260811 02:52:14.492900 18591 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.138.193:32779 every 8 connection(s)
I20260811 02:52:14.493942 18439 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786416707705494-12843-0/minicluster-data/ts-0/data/info.pb
I20260811 02:52:14.496172 12843 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDawEwS/build/debug/bin/kudu as pid 18439
I20260811 02:52:14.496327 12843 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786416707705494-12843-0/minicluster-data/ts-0/wal/instance
I20260811 02:52:14.496661 12843 external_mini_cluster.cc:1428] Setting key 9a70fd2623ce0a270e538d1221f786f2
I20260811 02:52:14.498987 12843 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDawEwS/build/debug/bin/kudu
/tmp/dist-test-taskDawEwS/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786416707705494-12843-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786416707705494-12843-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786416707705494-12843-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786416707705494-12843-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.138.194:0
--local_ip_for_outbound_sockets=127.12.138.194
--webserver_interface=127.12.138.194
--webserver_port=0
--tserver_master_addrs=127.12.138.254:37155
--builtin_ntp_servers=127.12.138.212:40813
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
I20260811 02:52:14.501266 18413 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:52:14.495960 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.138.193:49387 (local address 127.12.138.254:37155)
0811 02:52:14.496158 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0811 02:52:14.496164 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 02:52:14.497493 (+ 1329us) server_negotiation.cc:407] Connection header received
0811 02:52:14.499318 (+ 1825us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:52:14.499321 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:52:14.499356 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:52:14.499428 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:52:14.500688 (+ 1260us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:52:14.500691 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:52:14.500694 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 02:52:14.500707 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:52:14.500728 (+ 21us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 02:52:14.500739 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:52:14.500827 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0811 02:52:14.501051 (+ 224us) server_negotiation.cc:299] Negotiation successful
0811 02:52:14.501101 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260811 02:52:14.502437 18596 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:52:14.496399 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.138.254:37155 (local address 127.12.138.193:49387)
0811 02:52:14.497331 (+ 932us) negotiation.cc:107] Waiting for socket to connect
0811 02:52:14.497375 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0811 02:52:14.499028 (+ 1653us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:52:14.499618 (+ 590us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:52:14.499630 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:52:14.499758 (+ 128us) client_negotiation.cc:189] Negotiated authn=SASL
0811 02:52:14.500364 (+ 606us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 02:52:14.500368 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0811 02:52:14.500428 (+ 60us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 02:52:14.500520 (+ 92us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 02:52:14.500877 (+ 357us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 02:52:14.500886 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 02:52:14.500893 (+ 7us) client_negotiation.cc:769] Sending connection context
0811 02:52:14.501527 (+ 634us) client_negotiation.cc:240] Negotiation successful
0811 02:52:14.501721 (+ 194us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":297,"thread_start_us":124,"threads_started":1}
I20260811 02:52:14.503624 18594 heartbeater.cc:347] Connected to a master server at 127.12.138.254:37155
I20260811 02:52:14.503850 18594 heartbeater.cc:464] Registering TS with master...
I20260811 02:52:14.504256 18594 heartbeater.cc:511] Master 127.12.138.254:37155 requested a full tablet report, sending...
I20260811 02:52:14.505754 18366 ts_manager.cc:194] Registered new tserver with Master: ea24344f7c15420595cefc09fb128be0 (127.12.138.193:32779)
W20260811 02:52:14.618291 18597 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 02:52:14.618626 18597 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 02:52:14.618710 18597 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 02:52:14.622900 18597 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 02:52:14.622997 18597 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 02:52:14.623097 18597 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.138.194
I20260811 02:52:14.628253 18597 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.138.212:40813
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786416707705494-12843-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786416707705494-12843-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.138.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786416707705494-12843-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.12.138.194
--webserver_port=0
--tserver_master_addrs=127.12.138.254:37155
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18597
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.138.194
--log_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786416707705494-12843-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 02:09:49 UTC on ccycloud.quasar-fsnqqc.root.comops.site
build id 4579570
I20260811 02:52:14.629547 18597 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 02:52:14.631116 18597 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 02:52:14.641861 18602 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:52:14.641973 18603 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:52:14.641979 18597 server_base.cc:1034] running on GCE node
W20260811 02:52:14.642254 18605 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:52:14.642719 18597 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 02:52:14.643433 18597 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 02:52:14.644637 18597 hybrid_clock.cc:648] HybridClock initialized: now 1786416734644598 us; error 60 us; skew 500 ppm
I20260811 02:52:14.646903 18597 webserver.cc:460] Webserver started at http://127.12.138.194:32777/ using document root <none> and password file <none>
I20260811 02:52:14.647565 18597 fs_manager.cc:359] Metadata directory not provided
I20260811 02:52:14.647634 18597 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 02:52:14.647837 18597 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 02:52:14.650221 18597 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786416707705494-12843-0/minicluster-data/ts-1/data/instance:
uuid: "bb17c62c540d492684f8a297631ce5ba"
format_stamp: "Formatted at 2026-08-11 02:52:14 on dist-test-slave-9gcw"
server_key: "0972c3d1cc1b140a306ce91198ccbcb0"
server_key_iv: "a5a6a1da82e5dea2a4f897074b287797"
server_key_version: "encryptionkey@0"
I20260811 02:52:14.650792 18597 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786416707705494-12843-0/minicluster-data/ts-1/wal/instance:
uuid: "bb17c62c540d492684f8a297631ce5ba"
format_stamp: "Formatted at 2026-08-11 02:52:14 on dist-test-slave-9gcw"
server_key: "0972c3d1cc1b140a306ce91198ccbcb0"
server_key_iv: "a5a6a1da82e5dea2a4f897074b287797"
server_key_version: "encryptionkey@0"
I20260811 02:52:14.654883 18597 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260811 02:52:14.657909 18612 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 02:52:14.659286 18597 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260811 02:52:14.659423 18597 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786416707705494-12843-0/minicluster-data/ts-1/data,/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786416707705494-12843-0/minicluster-data/ts-1/wal
uuid: "bb17c62c540d492684f8a297631ce5ba"
format_stamp: "Formatted at 2026-08-11 02:52:14 on dist-test-slave-9gcw"
server_key: "0972c3d1cc1b140a306ce91198ccbcb0"
server_key_iv: "a5a6a1da82e5dea2a4f897074b287797"
server_key_version: "encryptionkey@0"
I20260811 02:52:14.659554 18597 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786416707705494-12843-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786416707705494-12843-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786416707705494-12843-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 02:52:14.664489 18597 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 02:52:14.665174 18597 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 02:52:14.665376 18597 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 02:52:14.665964 18597 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 02:52:14.667053 18597 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 02:52:14.667102 18597 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 02:52:14.667142 18597 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 02:52:14.667209 18597 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 02:52:14.677058 18597 rpc_server.cc:307] RPC server started. Bound to: 127.12.138.194:36887
I20260811 02:52:14.677224 18745 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.138.194:36887 every 8 connection(s)
I20260811 02:52:14.678196 18597 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786416707705494-12843-0/minicluster-data/ts-1/data/info.pb
I20260811 02:52:14.685026 18413 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:52:14.680313 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.138.194:42883 (local address 127.12.138.254:37155)
0811 02:52:14.680464 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0811 02:52:14.680468 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 02:52:14.681828 (+ 1360us) server_negotiation.cc:407] Connection header received
0811 02:52:14.683229 (+ 1401us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:52:14.683232 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:52:14.683272 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:52:14.683386 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:52:14.684610 (+ 1224us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:52:14.684613 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:52:14.684615 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 02:52:14.684627 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:52:14.684648 (+ 21us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 02:52:14.684656 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:52:14.684758 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0811 02:52:14.684872 (+ 114us) server_negotiation.cc:299] Negotiation successful
0811 02:52:14.684912 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260811 02:52:14.685956 18749 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:52:14.680752 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.138.254:37155 (local address 127.12.138.194:42883)
0811 02:52:14.681651 (+ 899us) negotiation.cc:107] Waiting for socket to connect
0811 02:52:14.681716 (+ 65us) client_negotiation.cc:174] Beginning negotiation
0811 02:52:14.683014 (+ 1298us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:52:14.683522 (+ 508us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:52:14.683529 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:52:14.683618 (+ 89us) client_negotiation.cc:189] Negotiated authn=SASL
0811 02:52:14.684311 (+ 693us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 02:52:14.684313 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0811 02:52:14.684353 (+ 40us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 02:52:14.684422 (+ 69us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 02:52:14.684736 (+ 314us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 02:52:14.684742 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 02:52:14.684746 (+ 4us) client_negotiation.cc:769] Sending connection context
0811 02:52:14.684892 (+ 146us) client_negotiation.cc:240] Negotiation successful
0811 02:52:14.685106 (+ 214us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":294,"thread_start_us":126,"threads_started":1}
I20260811 02:52:14.687103 12843 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDawEwS/build/debug/bin/kudu as pid 18597
I20260811 02:52:14.687186 18746 heartbeater.cc:347] Connected to a master server at 127.12.138.254:37155
I20260811 02:52:14.687249 12843 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786416707705494-12843-0/minicluster-data/ts-1/wal/instance
I20260811 02:52:14.687427 18746 heartbeater.cc:464] Registering TS with master...
I20260811 02:52:14.687629 12843 external_mini_cluster.cc:1428] Setting key 2358e9fbe6313e201a46c33bb2e6969a
I20260811 02:52:14.687933 18746 heartbeater.cc:511] Master 127.12.138.254:37155 requested a full tablet report, sending...
I20260811 02:52:14.689069 18361 ts_manager.cc:194] Registered new tserver with Master: bb17c62c540d492684f8a297631ce5ba (127.12.138.194:36887)
I20260811 02:52:14.689935 12843 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDawEwS/build/debug/bin/kudu
/tmp/dist-test-taskDawEwS/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786416707705494-12843-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786416707705494-12843-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786416707705494-12843-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786416707705494-12843-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.138.195:0
--local_ip_for_outbound_sockets=127.12.138.195
--webserver_interface=127.12.138.195
--webserver_port=0
--tserver_master_addrs=127.12.138.254:37155
--builtin_ntp_servers=127.12.138.212:40813
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260811 02:52:14.807837 18750 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 02:52:14.808107 18750 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 02:52:14.808147 18750 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 02:52:14.812269 18750 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 02:52:14.812331 18750 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 02:52:14.812424 18750 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.138.195
I20260811 02:52:14.817286 18750 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.138.212:40813
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786416707705494-12843-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786416707705494-12843-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.138.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786416707705494-12843-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.12.138.195
--webserver_port=0
--tserver_master_addrs=127.12.138.254:37155
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18750
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.138.195
--log_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786416707705494-12843-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 02:09:49 UTC on ccycloud.quasar-fsnqqc.root.comops.site
build id 4579570
I20260811 02:52:14.818426 18750 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 02:52:14.819847 18750 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 02:52:14.829648 18756 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:52:14.829690 18757 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:52:14.829831 18750 server_base.cc:1034] running on GCE node
W20260811 02:52:14.830006 18759 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:52:14.830410 18750 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 02:52:14.831142 18750 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 02:52:14.832353 18750 hybrid_clock.cc:648] HybridClock initialized: now 1786416734832317 us; error 51 us; skew 500 ppm
I20260811 02:52:14.834443 18750 webserver.cc:460] Webserver started at http://127.12.138.195:33837/ using document root <none> and password file <none>
I20260811 02:52:14.835041 18750 fs_manager.cc:359] Metadata directory not provided
I20260811 02:52:14.835137 18750 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 02:52:14.835392 18750 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 02:52:14.837598 18750 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786416707705494-12843-0/minicluster-data/ts-2/data/instance:
uuid: "ee761a20ff2a4535bfa01bd0b07a59fa"
format_stamp: "Formatted at 2026-08-11 02:52:14 on dist-test-slave-9gcw"
server_key: "639fcc8d82dc9e7a04f66d2e3132074d"
server_key_iv: "c2314564f8e71587caaa9e07698b218f"
server_key_version: "encryptionkey@0"
I20260811 02:52:14.838193 18750 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786416707705494-12843-0/minicluster-data/ts-2/wal/instance:
uuid: "ee761a20ff2a4535bfa01bd0b07a59fa"
format_stamp: "Formatted at 2026-08-11 02:52:14 on dist-test-slave-9gcw"
server_key: "639fcc8d82dc9e7a04f66d2e3132074d"
server_key_iv: "c2314564f8e71587caaa9e07698b218f"
server_key_version: "encryptionkey@0"
I20260811 02:52:14.842108 18750 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260811 02:52:14.844803 18765 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 02:52:14.846163 18750 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260811 02:52:14.846302 18750 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786416707705494-12843-0/minicluster-data/ts-2/data,/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786416707705494-12843-0/minicluster-data/ts-2/wal
uuid: "ee761a20ff2a4535bfa01bd0b07a59fa"
format_stamp: "Formatted at 2026-08-11 02:52:14 on dist-test-slave-9gcw"
server_key: "639fcc8d82dc9e7a04f66d2e3132074d"
server_key_iv: "c2314564f8e71587caaa9e07698b218f"
server_key_version: "encryptionkey@0"
I20260811 02:52:14.846421 18750 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786416707705494-12843-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786416707705494-12843-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786416707705494-12843-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 02:52:14.851897 18750 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 02:52:14.852495 18750 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 02:52:14.852638 18750 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 02:52:14.853134 18750 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 02:52:14.854120 18750 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 02:52:14.854166 18750 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 02:52:14.854208 18750 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 02:52:14.854224 18750 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 02:52:14.863620 18750 rpc_server.cc:307] RPC server started. Bound to: 127.12.138.195:35971
I20260811 02:52:14.863765 18902 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.138.195:35971 every 8 connection(s)
I20260811 02:52:14.864646 18750 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786416707705494-12843-0/minicluster-data/ts-2/data/info.pb
I20260811 02:52:14.867442 12843 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDawEwS/build/debug/bin/kudu as pid 18750
I20260811 02:52:14.867587 12843 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786416707705494-12843-0/minicluster-data/ts-2/wal/instance
I20260811 02:52:14.867935 12843 external_mini_cluster.cc:1428] Setting key 49b5e6a7a8f6b4502edc47041b182d67
I20260811 02:52:14.871954 18413 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:52:14.866722 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.138.195:44615 (local address 127.12.138.254:37155)
0811 02:52:14.866899 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0811 02:52:14.866904 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 02:52:14.868208 (+ 1304us) server_negotiation.cc:407] Connection header received
0811 02:52:14.870107 (+ 1899us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:52:14.870111 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:52:14.870169 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:52:14.870263 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:52:14.871546 (+ 1283us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:52:14.871549 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:52:14.871551 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 02:52:14.871562 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:52:14.871584 (+ 22us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 02:52:14.871592 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:52:14.871671 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0811 02:52:14.871806 (+ 135us) server_negotiation.cc:299] Negotiation successful
0811 02:52:14.871847 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260811 02:52:14.872962 18905 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:52:14.867131 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.138.254:37155 (local address 127.12.138.195:44615)
0811 02:52:14.868024 (+ 893us) negotiation.cc:107] Waiting for socket to connect
0811 02:52:14.868085 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0811 02:52:14.869846 (+ 1761us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:52:14.870497 (+ 651us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:52:14.870506 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:52:14.870696 (+ 190us) client_negotiation.cc:189] Negotiated authn=SASL
0811 02:52:14.871271 (+ 575us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 02:52:14.871273 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0811 02:52:14.871317 (+ 44us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 02:52:14.871389 (+ 72us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 02:52:14.871681 (+ 292us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 02:52:14.871688 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 02:52:14.871692 (+ 4us) client_negotiation.cc:769] Sending connection context
0811 02:52:14.871824 (+ 132us) client_negotiation.cc:240] Negotiation successful
0811 02:52:14.872073 (+ 249us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":308,"thread_start_us":130,"threads_started":1}
I20260811 02:52:14.874225 18903 heartbeater.cc:347] Connected to a master server at 127.12.138.254:37155
I20260811 02:52:14.874485 18903 heartbeater.cc:464] Registering TS with master...
I20260811 02:52:14.874976 18903 heartbeater.cc:511] Master 127.12.138.254:37155 requested a full tablet report, sending...
I20260811 02:52:14.876044 18361 ts_manager.cc:194] Registered new tserver with Master: ee761a20ff2a4535bfa01bd0b07a59fa (127.12.138.195:35971)
I20260811 02:52:14.883069 12843 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 02:52:14.887068 18413 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:52:14.885221 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53538 (local address 127.12.138.254:37155)
0811 02:52:14.885376 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0811 02:52:14.885381 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 02:52:14.885550 (+ 169us) server_negotiation.cc:407] Connection header received
0811 02:52:14.885731 (+ 181us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:52:14.885734 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:52:14.885780 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:52:14.885866 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:52:14.886633 (+ 767us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:52:14.886637 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:52:14.886640 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 02:52:14.886652 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:52:14.886680 (+ 28us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 02:52:14.886689 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:52:14.886785 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0811 02:52:14.886863 (+ 78us) server_negotiation.cc:299] Negotiation successful
0811 02:52:14.886911 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260811 02:52:14.890391 18361 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:53538:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 02:52:14.893590 18361 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 02:52:14.905740 18915 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:52:14.902365 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.138.194:36887 (local address 127.0.0.1:43848)
0811 02:52:14.903077 (+ 712us) negotiation.cc:107] Waiting for socket to connect
0811 02:52:14.903129 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0811 02:52:14.903257 (+ 128us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:52:14.904183 (+ 926us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:52:14.904190 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:52:14.904266 (+ 76us) client_negotiation.cc:189] Negotiated authn=SASL
0811 02:52:14.904889 (+ 623us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 02:52:14.904891 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0811 02:52:14.904911 (+ 20us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 02:52:14.904977 (+ 66us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 02:52:14.905347 (+ 370us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 02:52:14.905355 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 02:52:14.905361 (+ 6us) client_negotiation.cc:769] Sending connection context
0811 02:52:14.905479 (+ 118us) client_negotiation.cc:240] Negotiation successful
0811 02:52:14.905552 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":528,"thread_start_us":80,"threads_started":1}
I20260811 02:52:14.906239 18916 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:52:14.902749 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.138.193:32779 (local address 127.0.0.1:54776)
0811 02:52:14.903764 (+ 1015us) negotiation.cc:107] Waiting for socket to connect
0811 02:52:14.903777 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 02:52:14.903853 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:52:14.904871 (+ 1018us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:52:14.904875 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:52:14.904891 (+ 16us) client_negotiation.cc:189] Negotiated authn=SASL
0811 02:52:14.905445 (+ 554us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 02:52:14.905449 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0811 02:52:14.905463 (+ 14us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 02:52:14.905494 (+ 31us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 02:52:14.905981 (+ 487us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 02:52:14.905985 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 02:52:14.905988 (+ 3us) client_negotiation.cc:769] Sending connection context
0811 02:52:14.906053 (+ 65us) client_negotiation.cc:240] Negotiation successful
0811 02:52:14.906097 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":958,"thread_start_us":100,"threads_started":1}
I20260811 02:52:14.906498 18918 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:52:14.902438 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.138.195:35971 (local address 127.0.0.1:45440)
0811 02:52:14.903401 (+ 963us) negotiation.cc:107] Waiting for socket to connect
0811 02:52:14.903416 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 02:52:14.903496 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:52:14.904871 (+ 1375us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:52:14.904875 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:52:14.904891 (+ 16us) client_negotiation.cc:189] Negotiated authn=SASL
0811 02:52:14.905605 (+ 714us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 02:52:14.905609 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0811 02:52:14.905621 (+ 12us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 02:52:14.905645 (+ 24us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 02:52:14.906237 (+ 592us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 02:52:14.906240 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 02:52:14.906243 (+ 3us) client_negotiation.cc:769] Sending connection context
0811 02:52:14.906337 (+ 94us) client_negotiation.cc:240] Negotiation successful
0811 02:52:14.906374 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":902,"thread_start_us":111,"threads_started":1}
I20260811 02:52:14.906692 18921 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:52:14.903172 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54776 (local address 127.12.138.193:32779)
0811 02:52:14.903608 (+ 436us) server_negotiation.cc:206] Beginning negotiation
0811 02:52:14.903617 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 02:52:14.903951 (+ 334us) server_negotiation.cc:407] Connection header received
0811 02:52:14.904008 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:52:14.904014 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:52:14.904165 (+ 151us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:52:14.904286 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:52:14.905669 (+ 1383us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:52:14.905686 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:52:14.905701 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 02:52:14.905715 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:52:14.905774 (+ 59us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 02:52:14.905809 (+ 35us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:52:14.905940 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0811 02:52:14.906461 (+ 521us) server_negotiation.cc:299] Negotiation successful
0811 02:52:14.906540 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":354,"thread_start_us":151,"threads_started":1}
I20260811 02:52:14.906896 18919 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:52:14.903169 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45440 (local address 127.12.138.195:35971)
0811 02:52:14.903530 (+ 361us) server_negotiation.cc:206] Beginning negotiation
0811 02:52:14.903543 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0811 02:52:14.903570 (+ 27us) server_negotiation.cc:407] Connection header received
0811 02:52:14.903630 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:52:14.903638 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:52:14.903854 (+ 216us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:52:14.903994 (+ 140us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:52:14.905807 (+ 1813us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:52:14.905814 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:52:14.905829 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 02:52:14.906042 (+ 213us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:52:14.906092 (+ 50us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 02:52:14.906132 (+ 40us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:52:14.906254 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0811 02:52:14.906524 (+ 270us) server_negotiation.cc:299] Negotiation successful
0811 02:52:14.906742 (+ 218us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":252,"thread_start_us":107,"threads_started":1}
I20260811 02:52:14.907035 18917 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:52:14.902498 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43848 (local address 127.12.138.194:36887)
0811 02:52:14.903737 (+ 1239us) server_negotiation.cc:206] Beginning negotiation
0811 02:52:14.903751 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0811 02:52:14.903778 (+ 27us) server_negotiation.cc:407] Connection header received
0811 02:52:14.903834 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:52:14.903840 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:52:14.904026 (+ 186us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:52:14.904183 (+ 157us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:52:14.905109 (+ 926us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:52:14.905115 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:52:14.905127 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 02:52:14.905138 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:52:14.905217 (+ 79us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 02:52:14.905254 (+ 37us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:52:14.905371 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0811 02:52:14.905952 (+ 581us) server_negotiation.cc:299] Negotiation successful
0811 02:52:14.906874 (+ 922us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1136,"thread_start_us":186,"threads_started":1}
I20260811 02:52:14.909647 18824 tablet_service.cc:1467] Processing CreateTablet for tablet f784309e8e28475eb52898fa17ac1fcb (DEFAULT_TABLE table=test-table [id=db2e837f8cc04583af64d9ae775118c4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 02:52:14.909649 18512 tablet_service.cc:1467] Processing CreateTablet for tablet f784309e8e28475eb52898fa17ac1fcb (DEFAULT_TABLE table=test-table [id=db2e837f8cc04583af64d9ae775118c4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 02:52:14.909652 18663 tablet_service.cc:1467] Processing CreateTablet for tablet f784309e8e28475eb52898fa17ac1fcb (DEFAULT_TABLE table=test-table [id=db2e837f8cc04583af64d9ae775118c4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 02:52:14.910750 18663 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f784309e8e28475eb52898fa17ac1fcb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 02:52:14.910774 18824 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f784309e8e28475eb52898fa17ac1fcb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 02:52:14.910774 18512 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f784309e8e28475eb52898fa17ac1fcb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 02:52:14.916958 18922 tablet_bootstrap.cc:492] T f784309e8e28475eb52898fa17ac1fcb P bb17c62c540d492684f8a297631ce5ba: Bootstrap starting.
I20260811 02:52:14.918419 18923 tablet_bootstrap.cc:492] T f784309e8e28475eb52898fa17ac1fcb P ea24344f7c15420595cefc09fb128be0: Bootstrap starting.
I20260811 02:52:14.919725 18922 tablet_bootstrap.cc:654] T f784309e8e28475eb52898fa17ac1fcb P bb17c62c540d492684f8a297631ce5ba: Neither blocks nor log segments found. Creating new log.
I20260811 02:52:14.920557 18923 tablet_bootstrap.cc:654] T f784309e8e28475eb52898fa17ac1fcb P ea24344f7c15420595cefc09fb128be0: Neither blocks nor log segments found. Creating new log.
I20260811 02:52:14.920634 18922 log.cc:824] T f784309e8e28475eb52898fa17ac1fcb P bb17c62c540d492684f8a297631ce5ba: Log is configured to *not* fsync() on all Append() calls
I20260811 02:52:14.921001 18925 tablet_bootstrap.cc:492] T f784309e8e28475eb52898fa17ac1fcb P ee761a20ff2a4535bfa01bd0b07a59fa: Bootstrap starting.
I20260811 02:52:14.921396 18923 log.cc:824] T f784309e8e28475eb52898fa17ac1fcb P ea24344f7c15420595cefc09fb128be0: Log is configured to *not* fsync() on all Append() calls
I20260811 02:52:14.922940 18922 tablet_bootstrap.cc:492] T f784309e8e28475eb52898fa17ac1fcb P bb17c62c540d492684f8a297631ce5ba: No bootstrap required, opened a new log
I20260811 02:52:14.923103 18925 tablet_bootstrap.cc:654] T f784309e8e28475eb52898fa17ac1fcb P ee761a20ff2a4535bfa01bd0b07a59fa: Neither blocks nor log segments found. Creating new log.
I20260811 02:52:14.923238 18922 ts_tablet_manager.cc:1397] T f784309e8e28475eb52898fa17ac1fcb P bb17c62c540d492684f8a297631ce5ba: Time spent bootstrapping tablet: real 0.006s user 0.006s sys 0.000s
I20260811 02:52:14.923525 18923 tablet_bootstrap.cc:492] T f784309e8e28475eb52898fa17ac1fcb P ea24344f7c15420595cefc09fb128be0: No bootstrap required, opened a new log
I20260811 02:52:14.923717 18923 ts_tablet_manager.cc:1397] T f784309e8e28475eb52898fa17ac1fcb P ea24344f7c15420595cefc09fb128be0: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260811 02:52:14.923988 18925 log.cc:824] T f784309e8e28475eb52898fa17ac1fcb P ee761a20ff2a4535bfa01bd0b07a59fa: Log is configured to *not* fsync() on all Append() calls
I20260811 02:52:14.928216 18922 raft_consensus.cc:348] T f784309e8e28475eb52898fa17ac1fcb P bb17c62c540d492684f8a297631ce5ba [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb17c62c540d492684f8a297631ce5ba" member_type: VOTER last_known_addr { host: "127.12.138.194" port: 36887 } } peers { permanent_uuid: "ee761a20ff2a4535bfa01bd0b07a59fa" member_type: VOTER last_known_addr { host: "127.12.138.195" port: 35971 } } peers { permanent_uuid: "ea24344f7c15420595cefc09fb128be0" member_type: VOTER last_known_addr { host: "127.12.138.193" port: 32779 } }
I20260811 02:52:14.928220 18923 raft_consensus.cc:348] T f784309e8e28475eb52898fa17ac1fcb P ea24344f7c15420595cefc09fb128be0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb17c62c540d492684f8a297631ce5ba" member_type: VOTER last_known_addr { host: "127.12.138.194" port: 36887 } } peers { permanent_uuid: "ee761a20ff2a4535bfa01bd0b07a59fa" member_type: VOTER last_known_addr { host: "127.12.138.195" port: 35971 } } peers { permanent_uuid: "ea24344f7c15420595cefc09fb128be0" member_type: VOTER last_known_addr { host: "127.12.138.193" port: 32779 } }
I20260811 02:52:14.928498 18923 raft_consensus.cc:374] T f784309e8e28475eb52898fa17ac1fcb P ea24344f7c15420595cefc09fb128be0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 02:52:14.928514 18925 tablet_bootstrap.cc:492] T f784309e8e28475eb52898fa17ac1fcb P ee761a20ff2a4535bfa01bd0b07a59fa: No bootstrap required, opened a new log
I20260811 02:52:14.928552 18923 raft_consensus.cc:729] T f784309e8e28475eb52898fa17ac1fcb P ea24344f7c15420595cefc09fb128be0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ea24344f7c15420595cefc09fb128be0, State: Initialized, Role: FOLLOWER
I20260811 02:52:14.928668 18922 raft_consensus.cc:374] T f784309e8e28475eb52898fa17ac1fcb P bb17c62c540d492684f8a297631ce5ba [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 02:52:14.928754 18925 ts_tablet_manager.cc:1397] T f784309e8e28475eb52898fa17ac1fcb P ee761a20ff2a4535bfa01bd0b07a59fa: Time spent bootstrapping tablet: real 0.008s user 0.005s sys 0.000s
I20260811 02:52:14.928788 18922 raft_consensus.cc:729] T f784309e8e28475eb52898fa17ac1fcb P bb17c62c540d492684f8a297631ce5ba [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bb17c62c540d492684f8a297631ce5ba, State: Initialized, Role: FOLLOWER
I20260811 02:52:14.929032 18923 consensus_queue.cc:260] T f784309e8e28475eb52898fa17ac1fcb P ea24344f7c15420595cefc09fb128be0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb17c62c540d492684f8a297631ce5ba" member_type: VOTER last_known_addr { host: "127.12.138.194" port: 36887 } } peers { permanent_uuid: "ee761a20ff2a4535bfa01bd0b07a59fa" member_type: VOTER last_known_addr { host: "127.12.138.195" port: 35971 } } peers { permanent_uuid: "ea24344f7c15420595cefc09fb128be0" member_type: VOTER last_known_addr { host: "127.12.138.193" port: 32779 } }
I20260811 02:52:14.929539 18922 consensus_queue.cc:260] T f784309e8e28475eb52898fa17ac1fcb P bb17c62c540d492684f8a297631ce5ba [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb17c62c540d492684f8a297631ce5ba" member_type: VOTER last_known_addr { host: "127.12.138.194" port: 36887 } } peers { permanent_uuid: "ee761a20ff2a4535bfa01bd0b07a59fa" member_type: VOTER last_known_addr { host: "127.12.138.195" port: 35971 } } peers { permanent_uuid: "ea24344f7c15420595cefc09fb128be0" member_type: VOTER last_known_addr { host: "127.12.138.193" port: 32779 } }
I20260811 02:52:14.930078 18923 ts_tablet_manager.cc:1428] T f784309e8e28475eb52898fa17ac1fcb P ea24344f7c15420595cefc09fb128be0: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260811 02:52:14.930454 18594 heartbeater.cc:503] Master 127.12.138.254:37155 was elected leader, sending a full tablet report...
I20260811 02:52:14.930616 18922 ts_tablet_manager.cc:1428] T f784309e8e28475eb52898fa17ac1fcb P bb17c62c540d492684f8a297631ce5ba: Time spent starting tablet: real 0.007s user 0.006s sys 0.000s
I20260811 02:52:14.930967 18746 heartbeater.cc:503] Master 127.12.138.254:37155 was elected leader, sending a full tablet report...
I20260811 02:52:14.934023 18925 raft_consensus.cc:348] T f784309e8e28475eb52898fa17ac1fcb P ee761a20ff2a4535bfa01bd0b07a59fa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb17c62c540d492684f8a297631ce5ba" member_type: VOTER last_known_addr { host: "127.12.138.194" port: 36887 } } peers { permanent_uuid: "ee761a20ff2a4535bfa01bd0b07a59fa" member_type: VOTER last_known_addr { host: "127.12.138.195" port: 35971 } } peers { permanent_uuid: "ea24344f7c15420595cefc09fb128be0" member_type: VOTER last_known_addr { host: "127.12.138.193" port: 32779 } }
I20260811 02:52:14.934288 18925 raft_consensus.cc:374] T f784309e8e28475eb52898fa17ac1fcb P ee761a20ff2a4535bfa01bd0b07a59fa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 02:52:14.934345 18925 raft_consensus.cc:729] T f784309e8e28475eb52898fa17ac1fcb P ee761a20ff2a4535bfa01bd0b07a59fa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ee761a20ff2a4535bfa01bd0b07a59fa, State: Initialized, Role: FOLLOWER
I20260811 02:52:14.934823 18925 consensus_queue.cc:260] T f784309e8e28475eb52898fa17ac1fcb P ee761a20ff2a4535bfa01bd0b07a59fa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb17c62c540d492684f8a297631ce5ba" member_type: VOTER last_known_addr { host: "127.12.138.194" port: 36887 } } peers { permanent_uuid: "ee761a20ff2a4535bfa01bd0b07a59fa" member_type: VOTER last_known_addr { host: "127.12.138.195" port: 35971 } } peers { permanent_uuid: "ea24344f7c15420595cefc09fb128be0" member_type: VOTER last_known_addr { host: "127.12.138.193" port: 32779 } }
I20260811 02:52:14.935420 18903 heartbeater.cc:503] Master 127.12.138.254:37155 was elected leader, sending a full tablet report...
I20260811 02:52:14.935760 18925 ts_tablet_manager.cc:1428] T f784309e8e28475eb52898fa17ac1fcb P ee761a20ff2a4535bfa01bd0b07a59fa: Time spent starting tablet: real 0.007s user 0.005s sys 0.000s
W20260811 02:52:14.995244 18595 tablet.cc:2369] T f784309e8e28475eb52898fa17ac1fcb P ea24344f7c15420595cefc09fb128be0: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 02:52:15.115846 18904 tablet.cc:2369] T f784309e8e28475eb52898fa17ac1fcb P ee761a20ff2a4535bfa01bd0b07a59fa: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 02:52:15.156280 18929 raft_consensus.cc:482] T f784309e8e28475eb52898fa17ac1fcb P ea24344f7c15420595cefc09fb128be0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 02:52:15.156549 18929 raft_consensus.cc:504] T f784309e8e28475eb52898fa17ac1fcb P ea24344f7c15420595cefc09fb128be0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb17c62c540d492684f8a297631ce5ba" member_type: VOTER last_known_addr { host: "127.12.138.194" port: 36887 } } peers { permanent_uuid: "ee761a20ff2a4535bfa01bd0b07a59fa" member_type: VOTER last_known_addr { host: "127.12.138.195" port: 35971 } } peers { permanent_uuid: "ea24344f7c15420595cefc09fb128be0" member_type: VOTER last_known_addr { host: "127.12.138.193" port: 32779 } }
I20260811 02:52:15.158207 18929 leader_election.cc:288] T f784309e8e28475eb52898fa17ac1fcb P ea24344f7c15420595cefc09fb128be0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bb17c62c540d492684f8a297631ce5ba (127.12.138.194:36887), ee761a20ff2a4535bfa01bd0b07a59fa (127.12.138.195:35971)
I20260811 02:52:15.160506 18919 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:52:15.158718 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.138.193:35403 (local address 127.12.138.195:35971)
0811 02:52:15.158852 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0811 02:52:15.158856 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 02:52:15.158976 (+ 120us) server_negotiation.cc:407] Connection header received
0811 02:52:15.159132 (+ 156us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:52:15.159136 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:52:15.159171 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:52:15.159277 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:52:15.160076 (+ 799us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:52:15.160080 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:52:15.160082 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 02:52:15.160093 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:52:15.160114 (+ 21us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 02:52:15.160129 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:52:15.160250 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0811 02:52:15.160319 (+ 69us) server_negotiation.cc:299] Negotiation successful
0811 02:52:15.160363 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260811 02:52:15.160526 18934 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:52:15.158498 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.138.195:35971 (local address 127.12.138.193:35403)
0811 02:52:15.158866 (+ 368us) negotiation.cc:107] Waiting for socket to connect
0811 02:52:15.158883 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 02:52:15.159019 (+ 136us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:52:15.159293 (+ 274us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:52:15.159296 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:52:15.159311 (+ 15us) client_negotiation.cc:189] Negotiated authn=SASL
0811 02:52:15.159912 (+ 601us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 02:52:15.159913 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0811 02:52:15.159924 (+ 11us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 02:52:15.159950 (+ 26us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 02:52:15.160234 (+ 284us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 02:52:15.160237 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 02:52:15.160241 (+ 4us) client_negotiation.cc:769] Sending connection context
0811 02:52:15.160323 (+ 82us) client_negotiation.cc:240] Negotiation successful
0811 02:52:15.160363 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":302,"thread_start_us":97,"threads_started":1}
I20260811 02:52:15.160547 18917 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:52:15.158563 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.138.193:42027 (local address 127.12.138.194:36887)
0811 02:52:15.158736 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0811 02:52:15.158741 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 02:52:15.159025 (+ 284us) server_negotiation.cc:407] Connection header received
0811 02:52:15.159140 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:52:15.159144 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:52:15.159179 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:52:15.159278 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:52:15.160162 (+ 884us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:52:15.160166 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:52:15.160168 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 02:52:15.160180 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:52:15.160200 (+ 20us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 02:52:15.160211 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:52:15.160318 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0811 02:52:15.160381 (+ 63us) server_negotiation.cc:299] Negotiation successful
0811 02:52:15.160418 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260811 02:52:15.160759 18933 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:52:15.158499 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.138.194:36887 (local address 127.12.138.193:42027)
0811 02:52:15.158920 (+ 421us) negotiation.cc:107] Waiting for socket to connect
0811 02:52:15.158932 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 02:52:15.159052 (+ 120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:52:15.159296 (+ 244us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:52:15.159299 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:52:15.159314 (+ 15us) client_negotiation.cc:189] Negotiated authn=SASL
0811 02:52:15.159993 (+ 679us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 02:52:15.159995 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0811 02:52:15.160006 (+ 11us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 02:52:15.160039 (+ 33us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 02:52:15.160296 (+ 257us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 02:52:15.160300 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 02:52:15.160303 (+ 3us) client_negotiation.cc:769] Sending connection context
0811 02:52:15.160374 (+ 71us) client_negotiation.cc:240] Negotiation successful
0811 02:52:15.160412 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":359,"mutex_wait_us":25,"thread_start_us":84,"threads_started":1}
I20260811 02:52:15.161186 18846 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f784309e8e28475eb52898fa17ac1fcb" candidate_uuid: "ea24344f7c15420595cefc09fb128be0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ee761a20ff2a4535bfa01bd0b07a59fa" is_pre_election: true
I20260811 02:52:15.161547 18846 raft_consensus.cc:2454] T f784309e8e28475eb52898fa17ac1fcb P ee761a20ff2a4535bfa01bd0b07a59fa [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ea24344f7c15420595cefc09fb128be0 in term 0.
I20260811 02:52:15.161752 18690 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f784309e8e28475eb52898fa17ac1fcb" candidate_uuid: "ea24344f7c15420595cefc09fb128be0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bb17c62c540d492684f8a297631ce5ba" is_pre_election: true
I20260811 02:52:15.162065 18690 raft_consensus.cc:2454] T f784309e8e28475eb52898fa17ac1fcb P bb17c62c540d492684f8a297631ce5ba [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ea24344f7c15420595cefc09fb128be0 in term 0.
I20260811 02:52:15.162056 18456 leader_election.cc:302] T f784309e8e28475eb52898fa17ac1fcb P ea24344f7c15420595cefc09fb128be0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ea24344f7c15420595cefc09fb128be0, ee761a20ff2a4535bfa01bd0b07a59fa; no voters:
I20260811 02:52:15.162319 18929 raft_consensus.cc:2781] T f784309e8e28475eb52898fa17ac1fcb P ea24344f7c15420595cefc09fb128be0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 02:52:15.162441 18929 raft_consensus.cc:482] T f784309e8e28475eb52898fa17ac1fcb P ea24344f7c15420595cefc09fb128be0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 02:52:15.162580 18929 raft_consensus.cc:3037] T f784309e8e28475eb52898fa17ac1fcb P ea24344f7c15420595cefc09fb128be0 [term 0 FOLLOWER]: Advancing to term 1
I20260811 02:52:15.163791 18929 raft_consensus.cc:504] T f784309e8e28475eb52898fa17ac1fcb P ea24344f7c15420595cefc09fb128be0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb17c62c540d492684f8a297631ce5ba" member_type: VOTER last_known_addr { host: "127.12.138.194" port: 36887 } } peers { permanent_uuid: "ee761a20ff2a4535bfa01bd0b07a59fa" member_type: VOTER last_known_addr { host: "127.12.138.195" port: 35971 } } peers { permanent_uuid: "ea24344f7c15420595cefc09fb128be0" member_type: VOTER last_known_addr { host: "127.12.138.193" port: 32779 } }
I20260811 02:52:15.164155 18929 leader_election.cc:288] T f784309e8e28475eb52898fa17ac1fcb P ea24344f7c15420595cefc09fb128be0 [CANDIDATE]: Term 1 election: Requested vote from peers bb17c62c540d492684f8a297631ce5ba (127.12.138.194:36887), ee761a20ff2a4535bfa01bd0b07a59fa (127.12.138.195:35971)
I20260811 02:52:15.164444 18690 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f784309e8e28475eb52898fa17ac1fcb" candidate_uuid: "ea24344f7c15420595cefc09fb128be0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bb17c62c540d492684f8a297631ce5ba"
I20260811 02:52:15.164474 18846 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f784309e8e28475eb52898fa17ac1fcb" candidate_uuid: "ea24344f7c15420595cefc09fb128be0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ee761a20ff2a4535bfa01bd0b07a59fa"
I20260811 02:52:15.164580 18846 raft_consensus.cc:3037] T f784309e8e28475eb52898fa17ac1fcb P ee761a20ff2a4535bfa01bd0b07a59fa [term 0 FOLLOWER]: Advancing to term 1
I20260811 02:52:15.164582 18690 raft_consensus.cc:3037] T f784309e8e28475eb52898fa17ac1fcb P bb17c62c540d492684f8a297631ce5ba [term 0 FOLLOWER]: Advancing to term 1
I20260811 02:52:15.165949 18690 raft_consensus.cc:2454] T f784309e8e28475eb52898fa17ac1fcb P bb17c62c540d492684f8a297631ce5ba [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ea24344f7c15420595cefc09fb128be0 in term 1.
I20260811 02:52:15.165949 18846 raft_consensus.cc:2454] T f784309e8e28475eb52898fa17ac1fcb P ee761a20ff2a4535bfa01bd0b07a59fa [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ea24344f7c15420595cefc09fb128be0 in term 1.
I20260811 02:52:15.166249 18456 leader_election.cc:302] T f784309e8e28475eb52898fa17ac1fcb P ea24344f7c15420595cefc09fb128be0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ea24344f7c15420595cefc09fb128be0, ee761a20ff2a4535bfa01bd0b07a59fa; no voters:
I20260811 02:52:15.166445 18929 raft_consensus.cc:2781] T f784309e8e28475eb52898fa17ac1fcb P ea24344f7c15420595cefc09fb128be0 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 02:52:15.166666 18929 raft_consensus.cc:686] T f784309e8e28475eb52898fa17ac1fcb P ea24344f7c15420595cefc09fb128be0 [term 1 LEADER]: Becoming Leader. State: Replica: ea24344f7c15420595cefc09fb128be0, State: Running, Role: LEADER
I20260811 02:52:15.166996 18929 consensus_queue.cc:237] T f784309e8e28475eb52898fa17ac1fcb P ea24344f7c15420595cefc09fb128be0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb17c62c540d492684f8a297631ce5ba" member_type: VOTER last_known_addr { host: "127.12.138.194" port: 36887 } } peers { permanent_uuid: "ee761a20ff2a4535bfa01bd0b07a59fa" member_type: VOTER last_known_addr { host: "127.12.138.195" port: 35971 } } peers { permanent_uuid: "ea24344f7c15420595cefc09fb128be0" member_type: VOTER last_known_addr { host: "127.12.138.193" port: 32779 } }
I20260811 02:52:15.170483 18366 catalog_manager.cc:5495] T f784309e8e28475eb52898fa17ac1fcb P ea24344f7c15420595cefc09fb128be0 reported cstate change: term changed from 0 to 1, leader changed from <none> to ea24344f7c15420595cefc09fb128be0 (127.12.138.193). New cstate: current_term: 1 leader_uuid: "ea24344f7c15420595cefc09fb128be0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb17c62c540d492684f8a297631ce5ba" member_type: VOTER last_known_addr { host: "127.12.138.194" port: 36887 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ee761a20ff2a4535bfa01bd0b07a59fa" member_type: VOTER last_known_addr { host: "127.12.138.195" port: 35971 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ea24344f7c15420595cefc09fb128be0" member_type: VOTER last_known_addr { host: "127.12.138.193" port: 32779 } health_report { overall_health: HEALTHY } } }
W20260811 02:52:15.179366 18747 tablet.cc:2369] T f784309e8e28475eb52898fa17ac1fcb P bb17c62c540d492684f8a297631ce5ba: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 02:52:15.202418 18921 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:52:15.200815 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54784 (local address 127.12.138.193:32779)
0811 02:52:15.200965 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0811 02:52:15.200969 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 02:52:15.200984 (+ 15us) server_negotiation.cc:407] Connection header received
0811 02:52:15.201105 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:52:15.201107 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:52:15.201195 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:52:15.201269 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:52:15.202027 (+ 758us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:52:15.202030 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:52:15.202032 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 02:52:15.202042 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:52:15.202061 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 02:52:15.202068 (+ 7us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:52:15.202145 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0811 02:52:15.202209 (+ 64us) server_negotiation.cc:299] Negotiation successful
0811 02:52:15.202241 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260811 02:52:15.208608 18690 raft_consensus.cc:1261] T f784309e8e28475eb52898fa17ac1fcb P bb17c62c540d492684f8a297631ce5ba [term 1 FOLLOWER]: Refusing update from remote peer ea24344f7c15420595cefc09fb128be0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 02:52:15.208607 18846 raft_consensus.cc:1261] T f784309e8e28475eb52898fa17ac1fcb P ee761a20ff2a4535bfa01bd0b07a59fa [term 1 FOLLOWER]: Refusing update from remote peer ea24344f7c15420595cefc09fb128be0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 02:52:15.209445 18935 consensus_queue.cc:1035] T f784309e8e28475eb52898fa17ac1fcb P ea24344f7c15420595cefc09fb128be0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ee761a20ff2a4535bfa01bd0b07a59fa" member_type: VOTER last_known_addr { host: "127.12.138.195" port: 35971 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 02:52:15.209707 18929 consensus_queue.cc:1035] T f784309e8e28475eb52898fa17ac1fcb P ea24344f7c15420595cefc09fb128be0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bb17c62c540d492684f8a297631ce5ba" member_type: VOTER last_known_addr { host: "127.12.138.194" port: 36887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 02:52:15.217978 18940 mvcc.cc:205] Tried to move back new op lower bound from 7317162947406774272 to 7317162947246682112. Current Snapshot: MvccSnapshot[applied={T|T < 7317162947406774272}]
I20260811 02:52:15.217980 18939 mvcc.cc:205] Tried to move back new op lower bound from 7317162947406774272 to 7317162947246682112. Current Snapshot: MvccSnapshot[applied={T|T < 7317162947406774272}]
I20260811 02:52:15.225191 18366 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:53538:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 02:52:15.225380 18366 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:53538:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 02:52:15.227916 18366 catalog_manager.cc:5782] T 00000000000000000000000000000000 P f69489d1baf24b26b6785af7aa995e40: Sending DeleteTablet for 3 replicas of tablet f784309e8e28475eb52898fa17ac1fcb
I20260811 02:52:15.228746 18824 tablet_service.cc:1514] Processing DeleteTablet for tablet f784309e8e28475eb52898fa17ac1fcb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 02:52:15 UTC) from {username='slave'} at 127.0.0.1:45440
I20260811 02:52:15.228749 18663 tablet_service.cc:1514] Processing DeleteTablet for tablet f784309e8e28475eb52898fa17ac1fcb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 02:52:15 UTC) from {username='slave'} at 127.0.0.1:43848
I20260811 02:52:15.228826 18512 tablet_service.cc:1514] Processing DeleteTablet for tablet f784309e8e28475eb52898fa17ac1fcb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 02:52:15 UTC) from {username='slave'} at 127.0.0.1:54776
I20260811 02:52:15.229303 18951 tablet_replica.cc:330] T f784309e8e28475eb52898fa17ac1fcb P ee761a20ff2a4535bfa01bd0b07a59fa: stopping tablet replica
I20260811 02:52:15.229655 18951 raft_consensus.cc:2229] T f784309e8e28475eb52898fa17ac1fcb P ee761a20ff2a4535bfa01bd0b07a59fa [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 02:52:15.229988 18951 raft_consensus.cc:2258] T f784309e8e28475eb52898fa17ac1fcb P ee761a20ff2a4535bfa01bd0b07a59fa [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 02:52:15.230031 12843 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDawEwS/build/debug/bin/kudu with pid 18439
I20260811 02:52:15.230367 18949 tablet_replica.cc:330] T f784309e8e28475eb52898fa17ac1fcb P bb17c62c540d492684f8a297631ce5ba: stopping tablet replica
I20260811 02:52:15.230744 18949 raft_consensus.cc:2229] T f784309e8e28475eb52898fa17ac1fcb P bb17c62c540d492684f8a297631ce5ba [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 02:52:15.231112 18949 raft_consensus.cc:2258] T f784309e8e28475eb52898fa17ac1fcb P bb17c62c540d492684f8a297631ce5ba [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 02:52:15.233063 18951 ts_tablet_manager.cc:1905] T f784309e8e28475eb52898fa17ac1fcb P ee761a20ff2a4535bfa01bd0b07a59fa: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 02:52:15.235792 18949 ts_tablet_manager.cc:1905] T f784309e8e28475eb52898fa17ac1fcb P bb17c62c540d492684f8a297631ce5ba: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 02:52:15.236554 18951 ts_tablet_manager.cc:1918] T f784309e8e28475eb52898fa17ac1fcb P ee761a20ff2a4535bfa01bd0b07a59fa: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 02:52:15.236666 18951 log.cc:1196] T f784309e8e28475eb52898fa17ac1fcb P ee761a20ff2a4535bfa01bd0b07a59fa: Deleting WAL directory at /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786416707705494-12843-0/minicluster-data/ts-2/wal/wals/f784309e8e28475eb52898fa17ac1fcb
I20260811 02:52:15.237010 18951 ts_tablet_manager.cc:1939] T f784309e8e28475eb52898fa17ac1fcb P ee761a20ff2a4535bfa01bd0b07a59fa: Deleting consensus metadata
I20260811 02:52:15.238413 18346 catalog_manager.cc:4841] TS ee761a20ff2a4535bfa01bd0b07a59fa (127.12.138.195:35971): tablet f784309e8e28475eb52898fa17ac1fcb (table test-table [id=db2e837f8cc04583af64d9ae775118c4]) successfully deleted
I20260811 02:52:15.238643 18949 ts_tablet_manager.cc:1918] T f784309e8e28475eb52898fa17ac1fcb P bb17c62c540d492684f8a297631ce5ba: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 02:52:15.238742 18949 log.cc:1196] T f784309e8e28475eb52898fa17ac1fcb P bb17c62c540d492684f8a297631ce5ba: Deleting WAL directory at /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786416707705494-12843-0/minicluster-data/ts-1/wal/wals/f784309e8e28475eb52898fa17ac1fcb
I20260811 02:52:15.239095 18949 ts_tablet_manager.cc:1939] T f784309e8e28475eb52898fa17ac1fcb P bb17c62c540d492684f8a297631ce5ba: Deleting consensus metadata
I20260811 02:52:15.240105 18349 catalog_manager.cc:4841] TS bb17c62c540d492684f8a297631ce5ba (127.12.138.194:36887): tablet f784309e8e28475eb52898fa17ac1fcb (table test-table [id=db2e837f8cc04583af64d9ae775118c4]) successfully deleted
W20260811 02:52:15.242179 18348 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.12.138.193:32779 (error 108)
I20260811 02:52:15.242329 12843 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDawEwS/build/debug/bin/kudu with pid 18597
I20260811 02:52:15.246122 18916 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:52:15.245766 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.138.193:32779 (local address 127.0.0.1:54798)
0811 02:52:15.245937 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0811 02:52:15.246022 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.138.193:32779: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
W20260811 02:52:15.249346 18348 catalog_manager.cc:4584] TS ea24344f7c15420595cefc09fb128be0 (127.12.138.193:32779): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet f784309e8e28475eb52898fa17ac1fcb: Network error: Client connection negotiation failed: client connection to 127.12.138.193:32779: connect: Connection refused (error 111)
I20260811 02:52:15.251739 12843 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDawEwS/build/debug/bin/kudu with pid 18750
I20260811 02:52:15.260437 12843 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDawEwS/build/debug/bin/kudu with pid 18328
2026-08-11T02:52:15Z chronyd exiting
[ OK ] SecurityITest.TestDisableAuthenticationEncryption (1206 ms)
[ RUN ] SecurityITest.TestJwtMiniCluster
I20260811 02:52:15.286497 12843 mini_oidc.cc:152] Starting JWKS server
I20260811 02:52:15.288142 12843 webserver.cc:460] Webserver started at https://127.0.0.1:41271/ using document root <none> and password file <none>
I20260811 02:52:15.288350 12843 mini_oidc.cc:189] Starting OIDC Discovery server
I20260811 02:52:15.288643 12843 webserver.cc:460] Webserver started at http://127.0.0.1:33933/ using document root <none> and password file <none>
2026-08-11T02:52:15Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T02:52:15Z Disabled control of system clock
I20260811 02:52:15.299531 12843 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDawEwS/build/debug/bin/kudu
/tmp/dist-test-taskDawEwS/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786416707705494-12843-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786416707705494-12843-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786416707705494-12843-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786416707705494-12843-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.138.254:41349
--webserver_interface=127.12.138.254
--webserver_port=0
--builtin_ntp_servers=127.12.138.212:33859
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.12.138.254:41349
--enable_jwt_token_auth=true
--jwks_url=https://localhost:41271/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786416707705494-12843-0/testchainca.cert with env {}
W20260811 02:52:15.414497 18961 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 02:52:15.414783 18961 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 02:52:15.414824 18961 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 02:52:15.418835 18961 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 02:52:15.418895 18961 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 02:52:15.418910 18961 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 02:52:15.418928 18961 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 02:52:15.418944 18961 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260811 02:52:15.418962 18961 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260811 02:52:15.418982 18961 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:41271/jwks.json
I20260811 02:52:15.424401 18961 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.138.212:33859
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786416707705494-12843-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786416707705494-12843-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.12.138.254:41349
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.138.254:41349
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:41271/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786416707705494-12843-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.12.138.254
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786416707705494-12843-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18961
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786416707705494-12843-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 02:09:49 UTC on ccycloud.quasar-fsnqqc.root.comops.site
build id 4579570
I20260811 02:52:15.425465 18961 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 02:52:15.426462 18961 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 02:52:15.435264 18967 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:52:15.435302 18968 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:52:15.435271 18971 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:52:15.435326 18961 server_base.cc:1034] running on GCE node
I20260811 02:52:15.436089 18961 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 02:52:15.437330 18961 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 02:52:15.438517 18961 hybrid_clock.cc:648] HybridClock initialized: now 1786416735438500 us; error 31 us; skew 500 ppm
I20260811 02:52:15.440377 18961 webserver.cc:460] Webserver started at http://127.12.138.254:46385/ using document root <none> and password file <none>
I20260811 02:52:15.440975 18961 fs_manager.cc:359] Metadata directory not provided
I20260811 02:52:15.441074 18961 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 02:52:15.441345 18961 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 02:52:15.443430 18961 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786416707705494-12843-0/minicluster-data/master-0/data/instance:
uuid: "752a499cb60c4688b940e68dd81456ad"
format_stamp: "Formatted at 2026-08-11 02:52:15 on dist-test-slave-9gcw"
server_key: "ddd63a163978880a06ed5e16b811562f"
server_key_iv: "7e1b5aea1309ec2b13526c19116e437d"
server_key_version: "encryptionkey@0"
I20260811 02:52:15.444022 18961 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786416707705494-12843-0/minicluster-data/master-0/wal/instance:
uuid: "752a499cb60c4688b940e68dd81456ad"
format_stamp: "Formatted at 2026-08-11 02:52:15 on dist-test-slave-9gcw"
server_key: "ddd63a163978880a06ed5e16b811562f"
server_key_iv: "7e1b5aea1309ec2b13526c19116e437d"
server_key_version: "encryptionkey@0"
I20260811 02:52:15.447834 18961 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260811 02:52:15.450510 18978 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 02:52:15.451769 18961 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260811 02:52:15.452023 18961 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786416707705494-12843-0/minicluster-data/master-0/data,/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786416707705494-12843-0/minicluster-data/master-0/wal
uuid: "752a499cb60c4688b940e68dd81456ad"
format_stamp: "Formatted at 2026-08-11 02:52:15 on dist-test-slave-9gcw"
server_key: "ddd63a163978880a06ed5e16b811562f"
server_key_iv: "7e1b5aea1309ec2b13526c19116e437d"
server_key_version: "encryptionkey@0"
I20260811 02:52:15.452163 18961 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786416707705494-12843-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786416707705494-12843-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786416707705494-12843-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 02:52:15.467420 18961 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 02:52:15.468148 18961 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 02:52:15.468362 18961 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 02:52:15.476181 18961 rpc_server.cc:307] RPC server started. Bound to: 127.12.138.254:41349
I20260811 02:52:15.476186 19042 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.138.254:41349 every 8 connection(s)
I20260811 02:52:15.477206 18961 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786416707705494-12843-0/minicluster-data/master-0/data/info.pb
I20260811 02:52:15.480468 19044 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 02:52:15.486016 12843 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDawEwS/build/debug/bin/kudu as pid 18961
I20260811 02:52:15.486191 12843 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786416707705494-12843-0/minicluster-data/master-0/wal/instance
I20260811 02:52:15.486286 19044 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 752a499cb60c4688b940e68dd81456ad: Bootstrap starting.
I20260811 02:52:15.486553 12843 external_mini_cluster.cc:1428] Setting key f7fc103c1352a2202cc7743c923b7c05
I20260811 02:52:15.489073 19044 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 752a499cb60c4688b940e68dd81456ad: Neither blocks nor log segments found. Creating new log.
I20260811 02:52:15.489950 19044 log.cc:824] T 00000000000000000000000000000000 P 752a499cb60c4688b940e68dd81456ad: Log is configured to *not* fsync() on all Append() calls
I20260811 02:52:15.491961 19044 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 752a499cb60c4688b940e68dd81456ad: No bootstrap required, opened a new log
I20260811 02:52:15.495937 19044 raft_consensus.cc:348] T 00000000000000000000000000000000 P 752a499cb60c4688b940e68dd81456ad [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "752a499cb60c4688b940e68dd81456ad" member_type: VOTER last_known_addr { host: "127.12.138.254" port: 41349 } }
I20260811 02:52:15.496155 19047 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:52:15.487949 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49740 (local address 127.12.138.254:41349)
0811 02:52:15.488633 (+ 684us) server_negotiation.cc:206] Beginning negotiation
0811 02:52:15.488647 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0811 02:52:15.488692 (+ 45us) server_negotiation.cc:407] Connection header received
0811 02:52:15.489441 (+ 749us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:52:15.489461 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:52:15.489879 (+ 418us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:52:15.490228 (+ 349us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:52:15.490665 (+ 437us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:52:15.491652 (+ 987us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:52:15.492689 (+ 1037us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:52:15.493243 (+ 554us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:52:15.493414 (+ 171us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:52:15.494344 (+ 930us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:52:15.494349 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:52:15.494362 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 02:52:15.494769 (+ 407us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:52:15.494828 (+ 59us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 02:52:15.494850 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:52:15.494966 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0811 02:52:15.495117 (+ 151us) server_negotiation.cc:299] Negotiation successful
0811 02:52:15.495408 (+ 291us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":305,"thread_start_us":162,"threads_started":1}
I20260811 02:52:15.496202 19044 raft_consensus.cc:374] T 00000000000000000000000000000000 P 752a499cb60c4688b940e68dd81456ad [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 02:52:15.496270 19044 raft_consensus.cc:729] T 00000000000000000000000000000000 P 752a499cb60c4688b940e68dd81456ad [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 752a499cb60c4688b940e68dd81456ad, State: Initialized, Role: FOLLOWER
I20260811 02:52:15.496696 19044 consensus_queue.cc:260] T 00000000000000000000000000000000 P 752a499cb60c4688b940e68dd81456ad [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "752a499cb60c4688b940e68dd81456ad" member_type: VOTER last_known_addr { host: "127.12.138.254" port: 41349 } }
I20260811 02:52:15.496868 19044 raft_consensus.cc:388] T 00000000000000000000000000000000 P 752a499cb60c4688b940e68dd81456ad [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 02:52:15.496944 19044 raft_consensus.cc:482] T 00000000000000000000000000000000 P 752a499cb60c4688b940e68dd81456ad [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 02:52:15.497047 19044 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 752a499cb60c4688b940e68dd81456ad [term 0 FOLLOWER]: Advancing to term 1
I20260811 02:52:15.498113 19044 raft_consensus.cc:504] T 00000000000000000000000000000000 P 752a499cb60c4688b940e68dd81456ad [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "752a499cb60c4688b940e68dd81456ad" member_type: VOTER last_known_addr { host: "127.12.138.254" port: 41349 } }
I20260811 02:52:15.498458 19044 leader_election.cc:302] T 00000000000000000000000000000000 P 752a499cb60c4688b940e68dd81456ad [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 752a499cb60c4688b940e68dd81456ad; no voters:
I20260811 02:52:15.498780 19044 leader_election.cc:288] T 00000000000000000000000000000000 P 752a499cb60c4688b940e68dd81456ad [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 02:52:15.498943 19049 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 752a499cb60c4688b940e68dd81456ad [term 1 FOLLOWER]: Leader election won for term 1
I20260811 02:52:15.499123 12843 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 02:52:15.499176 19049 raft_consensus.cc:686] T 00000000000000000000000000000000 P 752a499cb60c4688b940e68dd81456ad [term 1 LEADER]: Becoming Leader. State: Replica: 752a499cb60c4688b940e68dd81456ad, State: Running, Role: LEADER
I20260811 02:52:15.499652 19049 consensus_queue.cc:237] T 00000000000000000000000000000000 P 752a499cb60c4688b940e68dd81456ad [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "752a499cb60c4688b940e68dd81456ad" member_type: VOTER last_known_addr { host: "127.12.138.254" port: 41349 } }
I20260811 02:52:15.500188 19044 sys_catalog.cc:564] T 00000000000000000000000000000000 P 752a499cb60c4688b940e68dd81456ad [sys.catalog]: configured and running, proceeding with master startup.
I20260811 02:52:15.501029 19050 sys_catalog.cc:455] T 00000000000000000000000000000000 P 752a499cb60c4688b940e68dd81456ad [sys.catalog]: SysCatalogTable state changed. Reason: New leader 752a499cb60c4688b940e68dd81456ad. Latest consensus state: current_term: 1 leader_uuid: "752a499cb60c4688b940e68dd81456ad" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "752a499cb60c4688b940e68dd81456ad" member_type: VOTER last_known_addr { host: "127.12.138.254" port: 41349 } } }
I20260811 02:52:15.501596 19051 sys_catalog.cc:455] T 00000000000000000000000000000000 P 752a499cb60c4688b940e68dd81456ad [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "752a499cb60c4688b940e68dd81456ad" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "752a499cb60c4688b940e68dd81456ad" member_type: VOTER last_known_addr { host: "127.12.138.254" port: 41349 } } }
I20260811 02:52:15.501724 19050 sys_catalog.cc:458] T 00000000000000000000000000000000 P 752a499cb60c4688b940e68dd81456ad [sys.catalog]: This master's current role is: LEADER
I20260811 02:52:15.501735 19051 sys_catalog.cc:458] T 00000000000000000000000000000000 P 752a499cb60c4688b940e68dd81456ad [sys.catalog]: This master's current role is: LEADER
I20260811 02:52:15.502209 19063 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 02:52:15.505548 19063 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 02:52:15.511410 19063 catalog_manager.cc:1347] Generated new cluster ID: b272353a078546ea9f1269e3de777c51
I20260811 02:52:15.511488 19063 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 02:52:15.553890 19063 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 02:52:15.555033 19063 catalog_manager.cc:1504] Loading token signing keys...
I20260811 02:52:15.561834 19063 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 752a499cb60c4688b940e68dd81456ad: Generated new TSK 0
I20260811 02:52:15.562556 19063 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260811 02:52:15.594882 19079 webserver.cc:533] Webserver: sslize failed: error:14094415:SSL routines:ssl3_read_bytes:sslv3 alert certificate expired
E20260811 02:52:15.595010 19079 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260811 02:52:15.595871 19047 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:52:15.577046 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49756 (local address 127.12.138.254:41349)
0811 02:52:15.577226 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0811 02:52:15.577232 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 02:52:15.577404 (+ 172us) server_negotiation.cc:407] Connection header received
0811 02:52:15.577603 (+ 199us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:52:15.577607 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:52:15.577670 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:52:15.577797 (+ 127us) server_negotiation.cc:226] Negotiated authn=JWT
0811 02:52:15.578032 (+ 235us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:52:15.578841 (+ 809us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:52:15.579733 (+ 892us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:52:15.580104 (+ 371us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:52:15.580231 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:52:15.580558 (+ 327us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0811 02:52:15.595369 (+ 14811us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:41271/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0811 02:52:15.595697 (+ 328us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:49756: Failed to load JWKS: Error downloading JWKS from 'https://localhost:41271/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"server-negotiator.queue_time_us":62}
W20260811 02:52:15.595970 19047 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:49756: Failed to load JWKS: Error downloading JWKS from 'https://localhost:41271/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
W20260811 02:52:15.597332 19077 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 02:52:15.576903 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.138.254:41349 (local address 127.0.0.1:49756)
0811 02:52:15.577319 (+ 416us) negotiation.cc:107] Waiting for socket to connect
0811 02:52:15.577332 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 02:52:15.577440 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:52:15.577819 (+ 379us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:52:15.577822 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:52:15.577829 (+ 7us) client_negotiation.cc:189] Negotiated authn=JWT
0811 02:52:15.577922 (+ 93us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:52:15.577927 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:52:15.578973 (+ 1046us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:52:15.578976 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:52:15.579613 (+ 637us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:52:15.579619 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:52:15.580229 (+ 610us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:52:15.580232 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:52:15.580309 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:52:15.580353 (+ 44us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0811 02:52:15.595619 (+ 15266us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:41271/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0811 02:52:15.595850 (+ 231us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.12.138.254:41349: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:41271/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"client-negotiator.queue_time_us":282,"thread_start_us":119,"threads_started":1}
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:597: Failure
Failed
Bad status: Runtime error: Could not connect to the cluster: Client connection negotiation failed: client connection to 127.12.138.254:41349: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:41271/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Google Test trace:
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:593: Valid JWT
I20260811 02:52:15.598049 12843 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDawEwS/build/debug/bin/kudu with pid 18961
2026-08-11T02:52:15Z chronyd exiting
I20260811 02:52:15.640655 12843 test_util.cc:175] -----------------------------------------------
I20260811 02:52:15.640785 12843 test_util.cc:176] Had fatal failures, leaving test files at /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786416707705494-12843-0
[ FAILED ] SecurityITest.TestJwtMiniCluster (356 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithInvalidCert
I20260811 02:52:15.642046 12843 mini_oidc.cc:152] Starting JWKS server
I20260811 02:52:15.642969 12843 webserver.cc:460] Webserver started at https://127.0.0.1:37557/ using document root <none> and password file <none>
I20260811 02:52:15.643105 12843 mini_oidc.cc:189] Starting OIDC Discovery server
I20260811 02:52:15.643386 12843 webserver.cc:460] Webserver started at http://127.0.0.1:37633/ using document root <none> and password file <none>
2026-08-11T02:52:15Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T02:52:15Z Disabled control of system clock
I20260811 02:52:15.653709 12843 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDawEwS/build/debug/bin/kudu
/tmp/dist-test-taskDawEwS/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786416707705494-12843-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786416707705494-12843-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786416707705494-12843-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786416707705494-12843-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.138.254:35567
--webserver_interface=127.12.138.254
--webserver_port=0
--builtin_ntp_servers=127.12.138.212:34655
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.12.138.254:35567
--enable_jwt_token_auth=true
--jwks_url=https://localhost:37557/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786416707705494-12843-0/testchainca.cert with env {}
W20260811 02:52:15.766620 19087 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 02:52:15.766914 19087 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 02:52:15.766954 19087 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 02:52:15.771029 19087 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 02:52:15.771095 19087 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 02:52:15.771109 19087 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 02:52:15.771126 19087 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 02:52:15.771195 19087 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260811 02:52:15.771224 19087 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260811 02:52:15.771253 19087 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:37557/jwks.json
I20260811 02:52:15.776747 19087 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.138.212:34655
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786416707705494-12843-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786416707705494-12843-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.12.138.254:35567
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.138.254:35567
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:37557/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786416707705494-12843-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.12.138.254
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786416707705494-12843-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19087
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786416707705494-12843-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 02:09:49 UTC on ccycloud.quasar-fsnqqc.root.comops.site
build id 4579570
I20260811 02:52:15.777941 19087 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 02:52:15.779006 19087 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 02:52:15.787887 19095 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:52:15.788156 19087 server_base.cc:1034] running on GCE node
W20260811 02:52:15.787865 19097 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:52:15.787905 19093 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:52:15.788918 19087 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 02:52:15.790099 19087 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 02:52:15.791270 19087 hybrid_clock.cc:648] HybridClock initialized: now 1786416735791235 us; error 43 us; skew 500 ppm
I20260811 02:52:15.793124 19087 webserver.cc:460] Webserver started at http://127.12.138.254:38523/ using document root <none> and password file <none>
I20260811 02:52:15.793781 19087 fs_manager.cc:359] Metadata directory not provided
I20260811 02:52:15.793850 19087 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 02:52:15.794082 19087 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 02:52:15.796217 19087 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786416707705494-12843-0/minicluster-data/master-0/data/instance:
uuid: "a630d6405f964bc59c6992fcfa799012"
format_stamp: "Formatted at 2026-08-11 02:52:15 on dist-test-slave-9gcw"
server_key: "474834bbf747b7f41ee3a3fc036e9298"
server_key_iv: "c0f0582b51e3329ea2b50022bc3b35c8"
server_key_version: "encryptionkey@0"
I20260811 02:52:15.796813 19087 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786416707705494-12843-0/minicluster-data/master-0/wal/instance:
uuid: "a630d6405f964bc59c6992fcfa799012"
format_stamp: "Formatted at 2026-08-11 02:52:15 on dist-test-slave-9gcw"
server_key: "474834bbf747b7f41ee3a3fc036e9298"
server_key_iv: "c0f0582b51e3329ea2b50022bc3b35c8"
server_key_version: "encryptionkey@0"
I20260811 02:52:15.800580 19087 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.003s
I20260811 02:52:15.803032 19104 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 02:52:15.804308 19087 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260811 02:52:15.804448 19087 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786416707705494-12843-0/minicluster-data/master-0/data,/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786416707705494-12843-0/minicluster-data/master-0/wal
uuid: "a630d6405f964bc59c6992fcfa799012"
format_stamp: "Formatted at 2026-08-11 02:52:15 on dist-test-slave-9gcw"
server_key: "474834bbf747b7f41ee3a3fc036e9298"
server_key_iv: "c0f0582b51e3329ea2b50022bc3b35c8"
server_key_version: "encryptionkey@0"
I20260811 02:52:15.804574 19087 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786416707705494-12843-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786416707705494-12843-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786416707705494-12843-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 02:52:15.831408 19087 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 02:52:15.832212 19087 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 02:52:15.832460 19087 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 02:52:15.839685 19087 rpc_server.cc:307] RPC server started. Bound to: 127.12.138.254:35567
I20260811 02:52:15.839712 19165 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.138.254:35567 every 8 connection(s)
I20260811 02:52:15.840714 19087 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786416707705494-12843-0/minicluster-data/master-0/data/info.pb
I20260811 02:52:15.844411 19167 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 02:52:15.850145 12843 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDawEwS/build/debug/bin/kudu as pid 19087
I20260811 02:52:15.850241 12843 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786416707705494-12843-0/minicluster-data/master-0/wal/instance
I20260811 02:52:15.850560 12843 external_mini_cluster.cc:1428] Setting key 6d621e91dd6d9dde34c989d62944b8b2
I20260811 02:52:15.850943 19167 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a630d6405f964bc59c6992fcfa799012: Bootstrap starting.
I20260811 02:52:15.853859 19167 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a630d6405f964bc59c6992fcfa799012: Neither blocks nor log segments found. Creating new log.
I20260811 02:52:15.854743 19167 log.cc:824] T 00000000000000000000000000000000 P a630d6405f964bc59c6992fcfa799012: Log is configured to *not* fsync() on all Append() calls
I20260811 02:52:15.856879 19167 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a630d6405f964bc59c6992fcfa799012: No bootstrap required, opened a new log
I20260811 02:52:15.860572 19167 raft_consensus.cc:348] T 00000000000000000000000000000000 P a630d6405f964bc59c6992fcfa799012 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a630d6405f964bc59c6992fcfa799012" member_type: VOTER last_known_addr { host: "127.12.138.254" port: 35567 } }
I20260811 02:52:15.860865 19167 raft_consensus.cc:374] T 00000000000000000000000000000000 P a630d6405f964bc59c6992fcfa799012 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 02:52:15.860894 19171 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:52:15.852069 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42174 (local address 127.12.138.254:35567)
0811 02:52:15.852797 (+ 728us) server_negotiation.cc:206] Beginning negotiation
0811 02:52:15.852818 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0811 02:52:15.852853 (+ 35us) server_negotiation.cc:407] Connection header received
0811 02:52:15.853587 (+ 734us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:52:15.853608 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:52:15.854072 (+ 464us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:52:15.854436 (+ 364us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:52:15.854910 (+ 474us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:52:15.855910 (+ 1000us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:52:15.856839 (+ 929us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:52:15.857441 (+ 602us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:52:15.857623 (+ 182us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:52:15.858601 (+ 978us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:52:15.858606 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:52:15.858618 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 02:52:15.859023 (+ 405us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:52:15.859080 (+ 57us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 02:52:15.859100 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:52:15.859212 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0811 02:52:15.859378 (+ 166us) server_negotiation.cc:299] Negotiation successful
0811 02:52:15.859638 (+ 260us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":373,"thread_start_us":165,"threads_started":1}
I20260811 02:52:15.860968 19167 raft_consensus.cc:729] T 00000000000000000000000000000000 P a630d6405f964bc59c6992fcfa799012 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a630d6405f964bc59c6992fcfa799012, State: Initialized, Role: FOLLOWER
I20260811 02:52:15.861555 19167 consensus_queue.cc:260] T 00000000000000000000000000000000 P a630d6405f964bc59c6992fcfa799012 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a630d6405f964bc59c6992fcfa799012" member_type: VOTER last_known_addr { host: "127.12.138.254" port: 35567 } }
I20260811 02:52:15.861752 19167 raft_consensus.cc:388] T 00000000000000000000000000000000 P a630d6405f964bc59c6992fcfa799012 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 02:52:15.861824 19167 raft_consensus.cc:482] T 00000000000000000000000000000000 P a630d6405f964bc59c6992fcfa799012 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 02:52:15.861919 19167 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a630d6405f964bc59c6992fcfa799012 [term 0 FOLLOWER]: Advancing to term 1
I20260811 02:52:15.863085 19167 raft_consensus.cc:504] T 00000000000000000000000000000000 P a630d6405f964bc59c6992fcfa799012 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a630d6405f964bc59c6992fcfa799012" member_type: VOTER last_known_addr { host: "127.12.138.254" port: 35567 } }
I20260811 02:52:15.863502 19167 leader_election.cc:302] T 00000000000000000000000000000000 P a630d6405f964bc59c6992fcfa799012 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a630d6405f964bc59c6992fcfa799012; no voters:
I20260811 02:52:15.863965 19167 leader_election.cc:288] T 00000000000000000000000000000000 P a630d6405f964bc59c6992fcfa799012 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 02:52:15.864044 19173 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a630d6405f964bc59c6992fcfa799012 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 02:52:15.864084 12843 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 02:52:15.864262 19173 raft_consensus.cc:686] T 00000000000000000000000000000000 P a630d6405f964bc59c6992fcfa799012 [term 1 LEADER]: Becoming Leader. State: Replica: a630d6405f964bc59c6992fcfa799012, State: Running, Role: LEADER
I20260811 02:52:15.864609 19173 consensus_queue.cc:237] T 00000000000000000000000000000000 P a630d6405f964bc59c6992fcfa799012 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a630d6405f964bc59c6992fcfa799012" member_type: VOTER last_known_addr { host: "127.12.138.254" port: 35567 } }
I20260811 02:52:15.865468 19167 sys_catalog.cc:564] T 00000000000000000000000000000000 P a630d6405f964bc59c6992fcfa799012 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 02:52:15.866609 19176 sys_catalog.cc:455] T 00000000000000000000000000000000 P a630d6405f964bc59c6992fcfa799012 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a630d6405f964bc59c6992fcfa799012. Latest consensus state: current_term: 1 leader_uuid: "a630d6405f964bc59c6992fcfa799012" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a630d6405f964bc59c6992fcfa799012" member_type: VOTER last_known_addr { host: "127.12.138.254" port: 35567 } } }
I20260811 02:52:15.866570 19174 sys_catalog.cc:455] T 00000000000000000000000000000000 P a630d6405f964bc59c6992fcfa799012 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a630d6405f964bc59c6992fcfa799012" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a630d6405f964bc59c6992fcfa799012" member_type: VOTER last_known_addr { host: "127.12.138.254" port: 35567 } } }
I20260811 02:52:15.866748 19176 sys_catalog.cc:458] T 00000000000000000000000000000000 P a630d6405f964bc59c6992fcfa799012 [sys.catalog]: This master's current role is: LEADER
I20260811 02:52:15.866814 19174 sys_catalog.cc:458] T 00000000000000000000000000000000 P a630d6405f964bc59c6992fcfa799012 [sys.catalog]: This master's current role is: LEADER
I20260811 02:52:15.867242 19184 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 02:52:15.870831 19184 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 02:52:15.877130 19184 catalog_manager.cc:1347] Generated new cluster ID: 08c45c580ac64fab9981c46f37d497a1
I20260811 02:52:15.877300 19184 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 02:52:15.910879 19184 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 02:52:15.911846 19184 catalog_manager.cc:1504] Loading token signing keys...
I20260811 02:52:15.927867 19184 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a630d6405f964bc59c6992fcfa799012: Generated new TSK 0
I20260811 02:52:15.928620 19184 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260811 02:52:15.952438 19202 webserver.cc:533] Webserver: sslize failed: error:14094418:SSL routines:ssl3_read_bytes:tlsv1 alert unknown ca
E20260811 02:52:15.952538 19202 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260811 02:52:15.953377 19171 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:52:15.937089 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42180 (local address 127.12.138.254:35567)
0811 02:52:15.937248 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0811 02:52:15.937254 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 02:52:15.937387 (+ 133us) server_negotiation.cc:407] Connection header received
0811 02:52:15.937521 (+ 134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:52:15.937525 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:52:15.937765 (+ 240us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:52:15.937908 (+ 143us) server_negotiation.cc:226] Negotiated authn=JWT
0811 02:52:15.938112 (+ 204us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:52:15.938944 (+ 832us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:52:15.939804 (+ 860us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:52:15.940186 (+ 382us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:52:15.940305 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:52:15.940616 (+ 311us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0811 02:52:15.952873 (+ 12257us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:37557/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0811 02:52:15.953219 (+ 346us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:42180: Failed to load JWKS: Error downloading JWKS from 'https://localhost:37557/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"server-negotiator.queue_time_us":82}
W20260811 02:52:15.953411 19201 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 02:52:15.936991 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.138.254:35567 (local address 127.0.0.1:42180)
0811 02:52:15.937286 (+ 295us) negotiation.cc:107] Waiting for socket to connect
0811 02:52:15.937303 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 02:52:15.937399 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:52:15.937902 (+ 503us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:52:15.937904 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:52:15.937910 (+ 6us) client_negotiation.cc:189] Negotiated authn=JWT
0811 02:52:15.937999 (+ 89us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:52:15.938004 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:52:15.939071 (+ 1067us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:52:15.939074 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:52:15.939685 (+ 611us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:52:15.939691 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:52:15.940327 (+ 636us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:52:15.940329 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:52:15.940401 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:52:15.940423 (+ 22us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0811 02:52:15.953074 (+ 12651us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:37557/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0811 02:52:15.953272 (+ 198us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.12.138.254:35567: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:37557/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"client-negotiator.queue_time_us":208,"thread_start_us":105,"threads_started":1}
W20260811 02:52:15.953454 19171 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:42180: Failed to load JWKS: Error downloading JWKS from 'https://localhost:37557/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
I20260811 02:52:15.954172 12843 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDawEwS/build/debug/bin/kudu with pid 19087
2026-08-11T02:52:15Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithInvalidCert (357 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithUntrustedCert
I20260811 02:52:15.999109 12843 mini_oidc.cc:152] Starting JWKS server
I20260811 02:52:16.000020 12843 webserver.cc:460] Webserver started at https://127.0.0.1:45711/ using document root <none> and password file <none>
I20260811 02:52:16.000170 12843 mini_oidc.cc:189] Starting OIDC Discovery server
I20260811 02:52:16.000497 12843 webserver.cc:460] Webserver started at http://127.0.0.1:33969/ using document root <none> and password file <none>
2026-08-11T02:52:16Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T02:52:16Z Disabled control of system clock
I20260811 02:52:16.011873 12843 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDawEwS/build/debug/bin/kudu
/tmp/dist-test-taskDawEwS/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786416707705494-12843-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786416707705494-12843-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786416707705494-12843-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786416707705494-12843-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.138.254:33775
--webserver_interface=127.12.138.254
--webserver_port=0
--builtin_ntp_servers=127.12.138.212:42751
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.12.138.254:33775
--enable_jwt_token_auth=true
--jwks_url=https://localhost:45711/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260811 02:52:16.127434 19209 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 02:52:16.127799 19209 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 02:52:16.127892 19209 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 02:52:16.132418 19209 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 02:52:16.132530 19209 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 02:52:16.132561 19209 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 02:52:16.132599 19209 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 02:52:16.132658 19209 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260811 02:52:16.132699 19209 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260811 02:52:16.132748 19209 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:45711/jwks.json
I20260811 02:52:16.138257 19209 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.138.212:42751
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786416707705494-12843-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786416707705494-12843-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.12.138.254:33775
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.138.254:33775
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:45711/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786416707705494-12843-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.12.138.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19209
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786416707705494-12843-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 02:09:49 UTC on ccycloud.quasar-fsnqqc.root.comops.site
build id 4579570
I20260811 02:52:16.139446 19209 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 02:52:16.140913 19209 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 02:52:16.149715 19217 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:52:16.149696 19214 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:52:16.149720 19215 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:52:16.150142 19209 server_base.cc:1034] running on GCE node
I20260811 02:52:16.150735 19209 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 02:52:16.151862 19209 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 02:52:16.153064 19209 hybrid_clock.cc:648] HybridClock initialized: now 1786416736153031 us; error 43 us; skew 500 ppm
I20260811 02:52:16.155018 19209 webserver.cc:460] Webserver started at http://127.12.138.254:46435/ using document root <none> and password file <none>
I20260811 02:52:16.155639 19209 fs_manager.cc:359] Metadata directory not provided
I20260811 02:52:16.155740 19209 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 02:52:16.155989 19209 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 02:52:16.158217 19209 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786416707705494-12843-0/minicluster-data/master-0/data/instance:
uuid: "df1b2c1fccbe4d239e7dc6f99e837921"
format_stamp: "Formatted at 2026-08-11 02:52:16 on dist-test-slave-9gcw"
server_key: "a793e44d03807cbd5494900ac31d7d90"
server_key_iv: "9d7c07aa002db3466594fceb5dd63a7d"
server_key_version: "encryptionkey@0"
I20260811 02:52:16.158805 19209 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786416707705494-12843-0/minicluster-data/master-0/wal/instance:
uuid: "df1b2c1fccbe4d239e7dc6f99e837921"
format_stamp: "Formatted at 2026-08-11 02:52:16 on dist-test-slave-9gcw"
server_key: "a793e44d03807cbd5494900ac31d7d90"
server_key_iv: "9d7c07aa002db3466594fceb5dd63a7d"
server_key_version: "encryptionkey@0"
I20260811 02:52:16.162510 19209 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.001s
I20260811 02:52:16.164937 19223 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 02:52:16.166194 19209 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260811 02:52:16.166347 19209 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786416707705494-12843-0/minicluster-data/master-0/data,/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786416707705494-12843-0/minicluster-data/master-0/wal
uuid: "df1b2c1fccbe4d239e7dc6f99e837921"
format_stamp: "Formatted at 2026-08-11 02:52:16 on dist-test-slave-9gcw"
server_key: "a793e44d03807cbd5494900ac31d7d90"
server_key_iv: "9d7c07aa002db3466594fceb5dd63a7d"
server_key_version: "encryptionkey@0"
I20260811 02:52:16.166463 19209 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786416707705494-12843-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786416707705494-12843-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786416707705494-12843-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 02:52:16.186082 19209 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 02:52:16.186767 19209 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 02:52:16.186988 19209 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 02:52:16.194376 19209 rpc_server.cc:307] RPC server started. Bound to: 127.12.138.254:33775
I20260811 02:52:16.194410 19289 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.138.254:33775 every 8 connection(s)
I20260811 02:52:16.195324 19209 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786416707705494-12843-0/minicluster-data/master-0/data/info.pb
I20260811 02:52:16.198381 12843 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDawEwS/build/debug/bin/kudu as pid 19209
I20260811 02:52:16.198531 12843 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786416707705494-12843-0/minicluster-data/master-0/wal/instance
I20260811 02:52:16.198678 19290 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 02:52:16.198904 12843 external_mini_cluster.cc:1428] Setting key 8db9ce6729aa56977ebeba20e93757ba
I20260811 02:52:16.205315 19290 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P df1b2c1fccbe4d239e7dc6f99e837921: Bootstrap starting.
I20260811 02:52:16.207479 19290 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P df1b2c1fccbe4d239e7dc6f99e837921: Neither blocks nor log segments found. Creating new log.
I20260811 02:52:16.208297 19290 log.cc:824] T 00000000000000000000000000000000 P df1b2c1fccbe4d239e7dc6f99e837921: Log is configured to *not* fsync() on all Append() calls
I20260811 02:52:16.208520 19293 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:52:16.200291 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47382 (local address 127.12.138.254:33775)
0811 02:52:16.201016 (+ 725us) server_negotiation.cc:206] Beginning negotiation
0811 02:52:16.201032 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0811 02:52:16.201067 (+ 35us) server_negotiation.cc:407] Connection header received
0811 02:52:16.202043 (+ 976us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:52:16.202063 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:52:16.202449 (+ 386us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:52:16.202784 (+ 335us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:52:16.203210 (+ 426us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:52:16.204168 (+ 958us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:52:16.205088 (+ 920us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:52:16.205681 (+ 593us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:52:16.205854 (+ 173us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:52:16.206757 (+ 903us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:52:16.206762 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:52:16.206773 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 02:52:16.207156 (+ 383us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:52:16.207210 (+ 54us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 02:52:16.207231 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:52:16.207355 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0811 02:52:16.207479 (+ 124us) server_negotiation.cc:299] Negotiation successful
0811 02:52:16.207737 (+ 258us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":296,"thread_start_us":146,"threads_started":1}
I20260811 02:52:16.210608 19290 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P df1b2c1fccbe4d239e7dc6f99e837921: No bootstrap required, opened a new log
I20260811 02:52:16.211548 12843 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 02:52:16.214423 19290 raft_consensus.cc:348] T 00000000000000000000000000000000 P df1b2c1fccbe4d239e7dc6f99e837921 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "df1b2c1fccbe4d239e7dc6f99e837921" member_type: VOTER last_known_addr { host: "127.12.138.254" port: 33775 } }
I20260811 02:52:16.214638 19290 raft_consensus.cc:374] T 00000000000000000000000000000000 P df1b2c1fccbe4d239e7dc6f99e837921 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 02:52:16.214713 19290 raft_consensus.cc:729] T 00000000000000000000000000000000 P df1b2c1fccbe4d239e7dc6f99e837921 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: df1b2c1fccbe4d239e7dc6f99e837921, State: Initialized, Role: FOLLOWER
I20260811 02:52:16.215118 19290 consensus_queue.cc:260] T 00000000000000000000000000000000 P df1b2c1fccbe4d239e7dc6f99e837921 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "df1b2c1fccbe4d239e7dc6f99e837921" member_type: VOTER last_known_addr { host: "127.12.138.254" port: 33775 } }
I20260811 02:52:16.215235 19290 raft_consensus.cc:388] T 00000000000000000000000000000000 P df1b2c1fccbe4d239e7dc6f99e837921 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 02:52:16.215372 19290 raft_consensus.cc:482] T 00000000000000000000000000000000 P df1b2c1fccbe4d239e7dc6f99e837921 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 02:52:16.215494 19290 raft_consensus.cc:3037] T 00000000000000000000000000000000 P df1b2c1fccbe4d239e7dc6f99e837921 [term 0 FOLLOWER]: Advancing to term 1
I20260811 02:52:16.216591 19290 raft_consensus.cc:504] T 00000000000000000000000000000000 P df1b2c1fccbe4d239e7dc6f99e837921 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "df1b2c1fccbe4d239e7dc6f99e837921" member_type: VOTER last_known_addr { host: "127.12.138.254" port: 33775 } }
I20260811 02:52:16.216930 19290 leader_election.cc:302] T 00000000000000000000000000000000 P df1b2c1fccbe4d239e7dc6f99e837921 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: df1b2c1fccbe4d239e7dc6f99e837921; no voters:
I20260811 02:52:16.217308 19290 leader_election.cc:288] T 00000000000000000000000000000000 P df1b2c1fccbe4d239e7dc6f99e837921 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 02:52:16.217420 19295 raft_consensus.cc:2781] T 00000000000000000000000000000000 P df1b2c1fccbe4d239e7dc6f99e837921 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 02:52:16.217686 19295 raft_consensus.cc:686] T 00000000000000000000000000000000 P df1b2c1fccbe4d239e7dc6f99e837921 [term 1 LEADER]: Becoming Leader. State: Replica: df1b2c1fccbe4d239e7dc6f99e837921, State: Running, Role: LEADER
I20260811 02:52:16.218022 19295 consensus_queue.cc:237] T 00000000000000000000000000000000 P df1b2c1fccbe4d239e7dc6f99e837921 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "df1b2c1fccbe4d239e7dc6f99e837921" member_type: VOTER last_known_addr { host: "127.12.138.254" port: 33775 } }
I20260811 02:52:16.218551 19290 sys_catalog.cc:564] T 00000000000000000000000000000000 P df1b2c1fccbe4d239e7dc6f99e837921 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 02:52:16.219725 19297 sys_catalog.cc:455] T 00000000000000000000000000000000 P df1b2c1fccbe4d239e7dc6f99e837921 [sys.catalog]: SysCatalogTable state changed. Reason: New leader df1b2c1fccbe4d239e7dc6f99e837921. Latest consensus state: current_term: 1 leader_uuid: "df1b2c1fccbe4d239e7dc6f99e837921" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "df1b2c1fccbe4d239e7dc6f99e837921" member_type: VOTER last_known_addr { host: "127.12.138.254" port: 33775 } } }
I20260811 02:52:16.219874 19297 sys_catalog.cc:458] T 00000000000000000000000000000000 P df1b2c1fccbe4d239e7dc6f99e837921 [sys.catalog]: This master's current role is: LEADER
I20260811 02:52:16.220163 19296 sys_catalog.cc:455] T 00000000000000000000000000000000 P df1b2c1fccbe4d239e7dc6f99e837921 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "df1b2c1fccbe4d239e7dc6f99e837921" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "df1b2c1fccbe4d239e7dc6f99e837921" member_type: VOTER last_known_addr { host: "127.12.138.254" port: 33775 } } }
I20260811 02:52:16.220333 19296 sys_catalog.cc:458] T 00000000000000000000000000000000 P df1b2c1fccbe4d239e7dc6f99e837921 [sys.catalog]: This master's current role is: LEADER
W20260811 02:52:16.223588 19313 catalog_manager.cc:1558] T 00000000000000000000000000000000 P df1b2c1fccbe4d239e7dc6f99e837921: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 02:52:16.223675 19313 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 02:52:16.223744 19304 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 02:52:16.224408 19304 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 02:52:16.229455 19304 catalog_manager.cc:1347] Generated new cluster ID: 474c0b748d9f4df192781e8de3f713af
I20260811 02:52:16.229522 19304 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 02:52:16.237946 19304 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 02:52:16.238986 19304 catalog_manager.cc:1504] Loading token signing keys...
I20260811 02:52:16.246996 19304 catalog_manager.cc:5868] T 00000000000000000000000000000000 P df1b2c1fccbe4d239e7dc6f99e837921: Generated new TSK 0
I20260811 02:52:16.247646 19304 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 02:52:16.264909 19293 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:52:16.259654 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47396 (local address 127.12.138.254:33775)
0811 02:52:16.259797 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0811 02:52:16.259803 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 02:52:16.259971 (+ 168us) server_negotiation.cc:407] Connection header received
0811 02:52:16.260111 (+ 140us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:52:16.260114 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:52:16.260170 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:52:16.260290 (+ 120us) server_negotiation.cc:226] Negotiated authn=JWT
0811 02:52:16.260509 (+ 219us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:52:16.261385 (+ 876us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:52:16.262249 (+ 864us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:52:16.262598 (+ 349us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:52:16.262707 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:52:16.263013 (+ 306us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0811 02:52:16.264520 (+ 1507us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:45711/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0811 02:52:16.264768 (+ 248us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:47396: Failed to load JWKS: Error downloading JWKS from 'https://localhost:45711/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"server-negotiator.queue_time_us":55}
W20260811 02:52:16.264988 19293 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:47396: Failed to load JWKS: Error downloading JWKS from 'https://localhost:45711/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
W20260811 02:52:16.266595 19322 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 02:52:16.259526 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.138.254:33775 (local address 127.0.0.1:47396)
0811 02:52:16.259886 (+ 360us) negotiation.cc:107] Waiting for socket to connect
0811 02:52:16.259899 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 02:52:16.259997 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:52:16.260294 (+ 297us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:52:16.260297 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:52:16.260305 (+ 8us) client_negotiation.cc:189] Negotiated authn=JWT
0811 02:52:16.260403 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:52:16.260409 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:52:16.261496 (+ 1087us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:52:16.261499 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:52:16.262127 (+ 628us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:52:16.262134 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:52:16.262732 (+ 598us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:52:16.262735 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:52:16.262813 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:52:16.262838 (+ 25us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0811 02:52:16.264686 (+ 1848us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:45711/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0811 02:52:16.266420 (+ 1734us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.12.138.254:33775: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:45711/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"client-negotiator.queue_time_us":256,"mutex_wait_us":1534,"thread_start_us":114,"threads_started":1}
I20260811 02:52:16.267606 12843 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDawEwS/build/debug/bin/kudu with pid 19209
E20260811 02:52:16.276768 19323 webserver.cc:533] Webserver: sslize failed:
E20260811 02:52:16.276870 19323 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
2026-08-11T02:52:16Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithUntrustedCert (311 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS
2026-08-11T02:52:16Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T02:52:16Z Disabled control of system clock
I20260811 02:52:16.320612 12843 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDawEwS/build/debug/bin/kudu
/tmp/dist-test-taskDawEwS/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786416707705494-12843-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786416707705494-12843-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786416707705494-12843-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786416707705494-12843-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.138.254:45577
--webserver_interface=127.12.138.254
--webserver_port=0
--builtin_ntp_servers=127.12.138.212:46875
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.12.138.254:45577
--enable_jwt_token_auth=true
--jwks_url=default.url
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786416707705494-12843-0/testchainca.cert with env {}
W20260811 02:52:16.434422 19328 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 02:52:16.434692 19328 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 02:52:16.434733 19328 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 02:52:16.438858 19328 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 02:52:16.438920 19328 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 02:52:16.438934 19328 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 02:52:16.438951 19328 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 02:52:16.438966 19328 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260811 02:52:16.439026 19328 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260811 02:52:16.439045 19328 flags.cc:425] Enabled experimental flag: --jwks_url=default.url
I20260811 02:52:16.444334 19328 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.138.212:46875
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786416707705494-12843-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786416707705494-12843-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.12.138.254:45577
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.138.254:45577
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=default.url
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786416707705494-12843-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.12.138.254
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786416707705494-12843-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19328
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786416707705494-12843-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 02:09:49 UTC on ccycloud.quasar-fsnqqc.root.comops.site
build id 4579570
I20260811 02:52:16.445431 19328 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 02:52:16.446527 19328 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 02:52:16.455519 19336 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:52:16.455511 19335 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:52:16.455701 19328 server_base.cc:1034] running on GCE node
W20260811 02:52:16.455535 19338 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:52:16.456419 19328 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 02:52:16.457651 19328 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 02:52:16.458832 19328 hybrid_clock.cc:648] HybridClock initialized: now 1786416736458805 us; error 31 us; skew 500 ppm
I20260811 02:52:16.460799 19328 webserver.cc:460] Webserver started at http://127.12.138.254:35557/ using document root <none> and password file <none>
I20260811 02:52:16.461474 19328 fs_manager.cc:359] Metadata directory not provided
I20260811 02:52:16.461541 19328 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 02:52:16.461763 19328 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 02:52:16.463851 19328 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786416707705494-12843-0/minicluster-data/master-0/data/instance:
uuid: "95d7f5b99e994110886d56138680a525"
format_stamp: "Formatted at 2026-08-11 02:52:16 on dist-test-slave-9gcw"
server_key: "b5c3f7e395691a79fe6b7251db4f5381"
server_key_iv: "dec7a6b5708d8814de088e52955a6224"
server_key_version: "encryptionkey@0"
I20260811 02:52:16.464443 19328 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786416707705494-12843-0/minicluster-data/master-0/wal/instance:
uuid: "95d7f5b99e994110886d56138680a525"
format_stamp: "Formatted at 2026-08-11 02:52:16 on dist-test-slave-9gcw"
server_key: "b5c3f7e395691a79fe6b7251db4f5381"
server_key_iv: "dec7a6b5708d8814de088e52955a6224"
server_key_version: "encryptionkey@0"
I20260811 02:52:16.468300 19328 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260811 02:52:16.470909 19344 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 02:52:16.472133 19328 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260811 02:52:16.472285 19328 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786416707705494-12843-0/minicluster-data/master-0/data,/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786416707705494-12843-0/minicluster-data/master-0/wal
uuid: "95d7f5b99e994110886d56138680a525"
format_stamp: "Formatted at 2026-08-11 02:52:16 on dist-test-slave-9gcw"
server_key: "b5c3f7e395691a79fe6b7251db4f5381"
server_key_iv: "dec7a6b5708d8814de088e52955a6224"
server_key_version: "encryptionkey@0"
I20260811 02:52:16.472409 19328 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786416707705494-12843-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786416707705494-12843-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786416707705494-12843-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 02:52:16.483956 19328 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 02:52:16.484663 19328 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 02:52:16.484879 19328 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 02:52:16.492339 19328 rpc_server.cc:307] RPC server started. Bound to: 127.12.138.254:45577
I20260811 02:52:16.492642 19406 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.138.254:45577 every 8 connection(s)
I20260811 02:52:16.493443 19328 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786416707705494-12843-0/minicluster-data/master-0/data/info.pb
I20260811 02:52:16.496378 19407 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 02:52:16.496948 12843 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDawEwS/build/debug/bin/kudu as pid 19328
I20260811 02:52:16.497048 12843 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786416707705494-12843-0/minicluster-data/master-0/wal/instance
I20260811 02:52:16.497426 12843 external_mini_cluster.cc:1428] Setting key 9fe9ddc9bf433053d441587bf16579ab
I20260811 02:52:16.502563 19407 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 95d7f5b99e994110886d56138680a525: Bootstrap starting.
I20260811 02:52:16.505273 19407 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 95d7f5b99e994110886d56138680a525: Neither blocks nor log segments found. Creating new log.
I20260811 02:52:16.506153 19407 log.cc:824] T 00000000000000000000000000000000 P 95d7f5b99e994110886d56138680a525: Log is configured to *not* fsync() on all Append() calls
I20260811 02:52:16.507061 19410 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:52:16.498831 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46798 (local address 127.12.138.254:45577)
0811 02:52:16.499443 (+ 612us) server_negotiation.cc:206] Beginning negotiation
0811 02:52:16.499458 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0811 02:52:16.499493 (+ 35us) server_negotiation.cc:407] Connection header received
0811 02:52:16.500149 (+ 656us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:52:16.500167 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:52:16.500548 (+ 381us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:52:16.500829 (+ 281us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:52:16.501244 (+ 415us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:52:16.502393 (+ 1149us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:52:16.503585 (+ 1192us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:52:16.504047 (+ 462us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:52:16.504166 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:52:16.505027 (+ 861us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:52:16.505032 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:52:16.505045 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 02:52:16.505572 (+ 527us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:52:16.505629 (+ 57us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 02:52:16.505651 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:52:16.505730 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0811 02:52:16.505892 (+ 162us) server_negotiation.cc:299] Negotiation successful
0811 02:52:16.506173 (+ 281us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":273,"thread_start_us":132,"threads_started":1}
I20260811 02:52:16.508165 19407 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 95d7f5b99e994110886d56138680a525: No bootstrap required, opened a new log
I20260811 02:52:16.509846 12843 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 02:52:16.512084 19407 raft_consensus.cc:348] T 00000000000000000000000000000000 P 95d7f5b99e994110886d56138680a525 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "95d7f5b99e994110886d56138680a525" member_type: VOTER last_known_addr { host: "127.12.138.254" port: 45577 } }
I20260811 02:52:16.512303 19407 raft_consensus.cc:374] T 00000000000000000000000000000000 P 95d7f5b99e994110886d56138680a525 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 02:52:16.512364 19407 raft_consensus.cc:729] T 00000000000000000000000000000000 P 95d7f5b99e994110886d56138680a525 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 95d7f5b99e994110886d56138680a525, State: Initialized, Role: FOLLOWER
I20260811 02:52:16.512816 19407 consensus_queue.cc:260] T 00000000000000000000000000000000 P 95d7f5b99e994110886d56138680a525 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "95d7f5b99e994110886d56138680a525" member_type: VOTER last_known_addr { host: "127.12.138.254" port: 45577 } }
I20260811 02:52:16.512960 19407 raft_consensus.cc:388] T 00000000000000000000000000000000 P 95d7f5b99e994110886d56138680a525 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 02:52:16.513049 19407 raft_consensus.cc:482] T 00000000000000000000000000000000 P 95d7f5b99e994110886d56138680a525 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 02:52:16.513190 19407 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 95d7f5b99e994110886d56138680a525 [term 0 FOLLOWER]: Advancing to term 1
I20260811 02:52:16.514082 19407 raft_consensus.cc:504] T 00000000000000000000000000000000 P 95d7f5b99e994110886d56138680a525 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "95d7f5b99e994110886d56138680a525" member_type: VOTER last_known_addr { host: "127.12.138.254" port: 45577 } }
I20260811 02:52:16.514417 19407 leader_election.cc:302] T 00000000000000000000000000000000 P 95d7f5b99e994110886d56138680a525 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 95d7f5b99e994110886d56138680a525; no voters:
I20260811 02:52:16.514727 19407 leader_election.cc:288] T 00000000000000000000000000000000 P 95d7f5b99e994110886d56138680a525 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 02:52:16.514866 19412 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 95d7f5b99e994110886d56138680a525 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 02:52:16.515113 19412 raft_consensus.cc:686] T 00000000000000000000000000000000 P 95d7f5b99e994110886d56138680a525 [term 1 LEADER]: Becoming Leader. State: Replica: 95d7f5b99e994110886d56138680a525, State: Running, Role: LEADER
I20260811 02:52:16.515427 19412 consensus_queue.cc:237] T 00000000000000000000000000000000 P 95d7f5b99e994110886d56138680a525 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "95d7f5b99e994110886d56138680a525" member_type: VOTER last_known_addr { host: "127.12.138.254" port: 45577 } }
I20260811 02:52:16.516135 19407 sys_catalog.cc:564] T 00000000000000000000000000000000 P 95d7f5b99e994110886d56138680a525 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 02:52:16.517398 19413 sys_catalog.cc:455] T 00000000000000000000000000000000 P 95d7f5b99e994110886d56138680a525 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "95d7f5b99e994110886d56138680a525" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "95d7f5b99e994110886d56138680a525" member_type: VOTER last_known_addr { host: "127.12.138.254" port: 45577 } } }
I20260811 02:52:16.517405 19414 sys_catalog.cc:455] T 00000000000000000000000000000000 P 95d7f5b99e994110886d56138680a525 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 95d7f5b99e994110886d56138680a525. Latest consensus state: current_term: 1 leader_uuid: "95d7f5b99e994110886d56138680a525" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "95d7f5b99e994110886d56138680a525" member_type: VOTER last_known_addr { host: "127.12.138.254" port: 45577 } } }
I20260811 02:52:16.517553 19413 sys_catalog.cc:458] T 00000000000000000000000000000000 P 95d7f5b99e994110886d56138680a525 [sys.catalog]: This master's current role is: LEADER
I20260811 02:52:16.517553 19414 sys_catalog.cc:458] T 00000000000000000000000000000000 P 95d7f5b99e994110886d56138680a525 [sys.catalog]: This master's current role is: LEADER
I20260811 02:52:16.517995 19424 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 02:52:16.520545 19424 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 02:52:16.526310 19424 catalog_manager.cc:1347] Generated new cluster ID: e84ebba1256249168a33a5ed06ac6a01
I20260811 02:52:16.526394 19424 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 02:52:16.541297 19424 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 02:52:16.542536 19424 catalog_manager.cc:1504] Loading token signing keys...
I20260811 02:52:16.552342 19424 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 95d7f5b99e994110886d56138680a525: Generated new TSK 0
I20260811 02:52:16.553072 19424 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 02:52:16.583039 19410 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:52:16.571930 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46800 (local address 127.12.138.254:45577)
0811 02:52:16.572057 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0811 02:52:16.572063 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 02:52:16.572233 (+ 170us) server_negotiation.cc:407] Connection header received
0811 02:52:16.572444 (+ 211us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:52:16.572446 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:52:16.572492 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:52:16.572606 (+ 114us) server_negotiation.cc:226] Negotiated authn=JWT
0811 02:52:16.572854 (+ 248us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:52:16.573468 (+ 614us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:52:16.574359 (+ 891us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:52:16.574594 (+ 235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:52:16.574703 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:52:16.575028 (+ 325us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0811 02:52:16.582600 (+ 7572us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0811 02:52:16.582893 (+ 293us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:46800: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"server-negotiator.queue_time_us":44}
W20260811 02:52:16.583128 19444 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 02:52:16.571851 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.138.254:45577 (local address 127.0.0.1:46800)
0811 02:52:16.572139 (+ 288us) negotiation.cc:107] Waiting for socket to connect
0811 02:52:16.572152 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 02:52:16.572281 (+ 129us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:52:16.572605 (+ 324us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:52:16.572609 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:52:16.572617 (+ 8us) client_negotiation.cc:189] Negotiated authn=JWT
0811 02:52:16.572729 (+ 112us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:52:16.572735 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:52:16.573584 (+ 849us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:52:16.573587 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:52:16.574230 (+ 643us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:52:16.574237 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:52:16.574730 (+ 493us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:52:16.574734 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:52:16.574828 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:52:16.574850 (+ 22us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0811 02:52:16.582797 (+ 7947us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0811 02:52:16.582953 (+ 156us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.12.138.254:45577: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"client-negotiator.queue_time_us":202,"thread_start_us":111,"threads_started":1}
W20260811 02:52:16.583133 19410 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:46800: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
I20260811 02:52:16.583752 12843 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDawEwS/build/debug/bin/kudu with pid 19328
2026-08-11T02:52:16Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS (296 ms)
[ RUN ] SecurityITest.TestWorldReadableKeytab
[ OK ] SecurityITest.TestWorldReadableKeytab (132 ms)
[ RUN ] SecurityITest.TestWorldReadablePrivateKey
[ OK ] SecurityITest.TestWorldReadablePrivateKey (130 ms)
[ RUN ] SecurityITest.TestCorruptKerberosCC
Loading random data
Initializing database '/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786416707705494-12843-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 02:52:16 dist-test-slave-9gcw krb5kdc[19457](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 02:52:16 dist-test-slave-9gcw krb5kdc[19457](info): set up 2 sockets
Aug 11 02:52:16 dist-test-slave-9gcw krb5kdc[19457](info): commencing operation
krb5kdc: starting...
W20260811 02:52:18.000435 12843 mini_kdc.cc:121] Time spent starting KDC: real 1.129s user 0.005s sys 0.002s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786416707705494-12843-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786416707705494-12843-0/krb5kdc/test-user.keytab.
Aug 11 02:52:18 dist-test-slave-9gcw krb5kdc[19457](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416738, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T02:52:18Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T02:52:18Z Disabled control of system clock
WARNING: no policy specified for kudu/127.12.138.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.138.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.138.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786416707705494-12843-0/krb5kdc/kudu_127.12.138.254.keytab.
Entry for principal kudu/127.12.138.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786416707705494-12843-0/krb5kdc/kudu_127.12.138.254.keytab.
I20260811 02:52:18.139276 12843 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDawEwS/build/debug/bin/kudu
/tmp/dist-test-taskDawEwS/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786416707705494-12843-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786416707705494-12843-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786416707705494-12843-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786416707705494-12843-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.138.254:38913
--webserver_interface=127.12.138.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786416707705494-12843-0/krb5kdc/kudu_127.12.138.254.keytab
--principal=kudu/127.12.138.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.138.212:41549
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.12.138.254:38913
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786416707705494-12843-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786416707705494-12843-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786416707705494-12843-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 02:52:18.270310 19475 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 02:52:18.270645 19475 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 02:52:18.270697 19475 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 02:52:18.275148 19475 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 02:52:18.275243 19475 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 02:52:18.275267 19475 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 02:52:18.275288 19475 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 02:52:18.275306 19475 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 02:52:18.281116 19475 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.138.212:41549
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786416707705494-12843-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786416707705494-12843-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.12.138.254:38913
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786416707705494-12843-0/krb5kdc/kudu_127.12.138.254.keytab
--principal=kudu/127.12.138.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.138.254:38913
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786416707705494-12843-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.12.138.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19475
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786416707705494-12843-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 02:09:49 UTC on ccycloud.quasar-fsnqqc.root.comops.site
build id 4579570
I20260811 02:52:18.282274 19475 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 02:52:18.283296 19475 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 02:52:18.292703 19481 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:52:18.292750 19483 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:52:18.292745 19475 server_base.cc:1034] running on GCE node
W20260811 02:52:18.292744 19480 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:52:18.293581 19475 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 02:52:18.294755 19475 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 02:52:18.295930 19475 hybrid_clock.cc:648] HybridClock initialized: now 1786416738295910 us; error 33 us; skew 500 ppm
Aug 11 02:52:18 dist-test-slave-9gcw krb5kdc[19457](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416738, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.138.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 02:52:18.298821 19475 init.cc:377] Logged in from keytab as kudu/127.12.138.254@KRBTEST.COM (short username kudu)
I20260811 02:52:18.299679 19475 webserver.cc:460] Webserver started at http://127.12.138.254:37587/ using document root <none> and password file <none>
I20260811 02:52:18.300258 19475 fs_manager.cc:359] Metadata directory not provided
I20260811 02:52:18.300323 19475 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 02:52:18.300498 19475 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 02:52:18.302608 19475 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786416707705494-12843-0/minicluster-data/master-0/data/instance:
uuid: "38d2c41aeec64ac5b659c4317e8c0530"
format_stamp: "Formatted at 2026-08-11 02:52:18 on dist-test-slave-9gcw"
server_key: "80a88c46ab5802b7d2474ce2b7ddeaac"
server_key_iv: "0dc197382862255303ad0197e204f288"
server_key_version: "encryptionkey@0"
I20260811 02:52:18.303174 19475 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786416707705494-12843-0/minicluster-data/master-0/wal/instance:
uuid: "38d2c41aeec64ac5b659c4317e8c0530"
format_stamp: "Formatted at 2026-08-11 02:52:18 on dist-test-slave-9gcw"
server_key: "80a88c46ab5802b7d2474ce2b7ddeaac"
server_key_iv: "0dc197382862255303ad0197e204f288"
server_key_version: "encryptionkey@0"
I20260811 02:52:18.307142 19475 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260811 02:52:18.309651 19492 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 02:52:18.310926 19475 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260811 02:52:18.311074 19475 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786416707705494-12843-0/minicluster-data/master-0/data,/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786416707705494-12843-0/minicluster-data/master-0/wal
uuid: "38d2c41aeec64ac5b659c4317e8c0530"
format_stamp: "Formatted at 2026-08-11 02:52:18 on dist-test-slave-9gcw"
server_key: "80a88c46ab5802b7d2474ce2b7ddeaac"
server_key_iv: "0dc197382862255303ad0197e204f288"
server_key_version: "encryptionkey@0"
I20260811 02:52:18.311192 19475 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786416707705494-12843-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786416707705494-12843-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786416707705494-12843-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 02:52:18.323468 19475 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 02:52:18.326575 19475 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 02:52:18.326814 19475 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 02:52:18.334210 19475 rpc_server.cc:307] RPC server started. Bound to: 127.12.138.254:38913
I20260811 02:52:18.334266 19551 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.138.254:38913 every 8 connection(s)
I20260811 02:52:18.335276 19475 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786416707705494-12843-0/minicluster-data/master-0/data/info.pb
I20260811 02:52:18.338526 19552 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 02:52:18.338927 12843 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDawEwS/build/debug/bin/kudu as pid 19475
I20260811 02:52:18.339087 12843 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786416707705494-12843-0/minicluster-data/master-0/wal/instance
I20260811 02:52:18.339406 12843 external_mini_cluster.cc:1428] Setting key aa82a66c8172289df86d66c89df7c086
I20260811 02:52:18.346282 19552 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 38d2c41aeec64ac5b659c4317e8c0530: Bootstrap starting.
I20260811 02:52:18.349267 19552 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 38d2c41aeec64ac5b659c4317e8c0530: Neither blocks nor log segments found. Creating new log.
Aug 11 02:52:18 dist-test-slave-9gcw krb5kdc[19457](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416738, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.12.138.254@KRBTEST.COM
I20260811 02:52:18.350090 19552 log.cc:824] T 00000000000000000000000000000000 P 38d2c41aeec64ac5b659c4317e8c0530: Log is configured to *not* fsync() on all Append() calls
I20260811 02:52:18.352430 19552 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 38d2c41aeec64ac5b659c4317e8c0530: No bootstrap required, opened a new log
I20260811 02:52:18.355041 19555 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:52:18.340926 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53550 (local address 127.12.138.254:38913)
0811 02:52:18.341348 (+ 422us) server_negotiation.cc:206] Beginning negotiation
0811 02:52:18.341372 (+ 24us) server_negotiation.cc:399] Waiting for connection header
0811 02:52:18.341409 (+ 37us) server_negotiation.cc:407] Connection header received
0811 02:52:18.342116 (+ 707us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:52:18.342134 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:52:18.342477 (+ 343us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:52:18.342764 (+ 287us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:52:18.343873 (+ 1109us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:52:18.344930 (+ 1057us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:52:18.346235 (+ 1305us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:52:18.346759 (+ 524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:52:18.346890 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:52:18.349992 (+ 3102us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:52:18.349997 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:52:18.350010 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:52:18.350038 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:52:18.352012 (+ 1974us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:52:18.352508 (+ 496us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:52:18.352513 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:52:18.352517 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:52:18.352679 (+ 162us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:52:18.352978 (+ 299us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:52:18.352982 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:52:18.352983 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:52:18.353475 (+ 492us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:52:18.353696 (+ 221us) server_negotiation.cc:1035] Waiting for connection context
0811 02:52:18.353988 (+ 292us) server_negotiation.cc:299] Negotiation successful
0811 02:52:18.354298 (+ 310us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":278,"thread_start_us":130,"threads_started":1}
I20260811 02:52:18.356410 19552 raft_consensus.cc:348] T 00000000000000000000000000000000 P 38d2c41aeec64ac5b659c4317e8c0530 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "38d2c41aeec64ac5b659c4317e8c0530" member_type: VOTER last_known_addr { host: "127.12.138.254" port: 38913 } }
I20260811 02:52:18.356709 19552 raft_consensus.cc:374] T 00000000000000000000000000000000 P 38d2c41aeec64ac5b659c4317e8c0530 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 02:52:18.356775 19552 raft_consensus.cc:729] T 00000000000000000000000000000000 P 38d2c41aeec64ac5b659c4317e8c0530 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 38d2c41aeec64ac5b659c4317e8c0530, State: Initialized, Role: FOLLOWER
I20260811 02:52:18.357292 19552 consensus_queue.cc:260] T 00000000000000000000000000000000 P 38d2c41aeec64ac5b659c4317e8c0530 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "38d2c41aeec64ac5b659c4317e8c0530" member_type: VOTER last_known_addr { host: "127.12.138.254" port: 38913 } }
I20260811 02:52:18.357470 19552 raft_consensus.cc:388] T 00000000000000000000000000000000 P 38d2c41aeec64ac5b659c4317e8c0530 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 02:52:18.357551 19552 raft_consensus.cc:482] T 00000000000000000000000000000000 P 38d2c41aeec64ac5b659c4317e8c0530 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 02:52:18.357633 19552 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 38d2c41aeec64ac5b659c4317e8c0530 [term 0 FOLLOWER]: Advancing to term 1
I20260811 02:52:18.358575 19552 raft_consensus.cc:504] T 00000000000000000000000000000000 P 38d2c41aeec64ac5b659c4317e8c0530 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "38d2c41aeec64ac5b659c4317e8c0530" member_type: VOTER last_known_addr { host: "127.12.138.254" port: 38913 } }
I20260811 02:52:18.358973 19552 leader_election.cc:302] T 00000000000000000000000000000000 P 38d2c41aeec64ac5b659c4317e8c0530 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 38d2c41aeec64ac5b659c4317e8c0530; no voters:
I20260811 02:52:18.359366 19552 leader_election.cc:288] T 00000000000000000000000000000000 P 38d2c41aeec64ac5b659c4317e8c0530 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 02:52:18.359447 19557 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 38d2c41aeec64ac5b659c4317e8c0530 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 02:52:18.359700 19557 raft_consensus.cc:686] T 00000000000000000000000000000000 P 38d2c41aeec64ac5b659c4317e8c0530 [term 1 LEADER]: Becoming Leader. State: Replica: 38d2c41aeec64ac5b659c4317e8c0530, State: Running, Role: LEADER
I20260811 02:52:18.360013 19557 consensus_queue.cc:237] T 00000000000000000000000000000000 P 38d2c41aeec64ac5b659c4317e8c0530 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "38d2c41aeec64ac5b659c4317e8c0530" member_type: VOTER last_known_addr { host: "127.12.138.254" port: 38913 } }
I20260811 02:52:18.360744 19552 sys_catalog.cc:564] T 00000000000000000000000000000000 P 38d2c41aeec64ac5b659c4317e8c0530 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 02:52:18.361809 19559 sys_catalog.cc:455] T 00000000000000000000000000000000 P 38d2c41aeec64ac5b659c4317e8c0530 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "38d2c41aeec64ac5b659c4317e8c0530" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "38d2c41aeec64ac5b659c4317e8c0530" member_type: VOTER last_known_addr { host: "127.12.138.254" port: 38913 } } }
I20260811 02:52:18.361996 19559 sys_catalog.cc:458] T 00000000000000000000000000000000 P 38d2c41aeec64ac5b659c4317e8c0530 [sys.catalog]: This master's current role is: LEADER
I20260811 02:52:18.362385 19558 sys_catalog.cc:455] T 00000000000000000000000000000000 P 38d2c41aeec64ac5b659c4317e8c0530 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 38d2c41aeec64ac5b659c4317e8c0530. Latest consensus state: current_term: 1 leader_uuid: "38d2c41aeec64ac5b659c4317e8c0530" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "38d2c41aeec64ac5b659c4317e8c0530" member_type: VOTER last_known_addr { host: "127.12.138.254" port: 38913 } } }
I20260811 02:52:18.362498 19558 sys_catalog.cc:458] T 00000000000000000000000000000000 P 38d2c41aeec64ac5b659c4317e8c0530 [sys.catalog]: This master's current role is: LEADER
W20260811 02:52:18.366000 19573 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 38d2c41aeec64ac5b659c4317e8c0530: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 02:52:18.366094 19573 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 02:52:18.366220 19566 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 02:52:18.367014 19566 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 02:52:18.373351 19566 catalog_manager.cc:1347] Generated new cluster ID: f10e078a31c94f63b21d803df082b5ca
I20260811 02:52:18.373426 19566 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 02:52:18.390172 19566 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 02:52:18.391532 19566 catalog_manager.cc:1504] Loading token signing keys...
I20260811 02:52:18.397895 19566 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 38d2c41aeec64ac5b659c4317e8c0530: Generated new TSK 0
I20260811 02:52:18.398749 19566 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.12.138.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.138.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.138.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786416707705494-12843-0/krb5kdc/kudu_127.12.138.193.keytab.
Entry for principal kudu/127.12.138.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786416707705494-12843-0/krb5kdc/kudu_127.12.138.193.keytab.
I20260811 02:52:18.436687 12843 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDawEwS/build/debug/bin/kudu
/tmp/dist-test-taskDawEwS/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786416707705494-12843-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786416707705494-12843-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786416707705494-12843-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786416707705494-12843-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.138.193:0
--local_ip_for_outbound_sockets=127.12.138.193
--webserver_interface=127.12.138.193
--webserver_port=0
--tserver_master_addrs=127.12.138.254:38913
--keytab_file=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786416707705494-12843-0/krb5kdc/kudu_127.12.138.193.keytab
--principal=kudu/127.12.138.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.138.212:41549
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786416707705494-12843-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786416707705494-12843-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786416707705494-12843-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 02:52:18.552146 19581 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 02:52:18.552477 19581 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 02:52:18.552553 19581 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 02:52:18.556625 19581 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 02:52:18.556717 19581 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 02:52:18.556811 19581 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.138.193
I20260811 02:52:18.562340 19581 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.138.212:41549
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786416707705494-12843-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786416707705494-12843-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786416707705494-12843-0/krb5kdc/kudu_127.12.138.193.keytab
--principal=kudu/127.12.138.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.138.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786416707705494-12843-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.12.138.193
--webserver_port=0
--tserver_master_addrs=127.12.138.254:38913
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19581
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.138.193
--log_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786416707705494-12843-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 02:09:49 UTC on ccycloud.quasar-fsnqqc.root.comops.site
build id 4579570
I20260811 02:52:18.563452 19581 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 02:52:18.564467 19581 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 02:52:18.574564 19592 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:52:18.574532 19589 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:52:18.574630 19581 server_base.cc:1034] running on GCE node
W20260811 02:52:18.574548 19590 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:52:18.575212 19581 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 02:52:18.575927 19581 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 02:52:18.577111 19581 hybrid_clock.cc:648] HybridClock initialized: now 1786416738577081 us; error 44 us; skew 500 ppm
Aug 11 02:52:18 dist-test-slave-9gcw krb5kdc[19457](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416738, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.138.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 02:52:18.580103 19581 init.cc:377] Logged in from keytab as kudu/127.12.138.193@KRBTEST.COM (short username kudu)
I20260811 02:52:18.581004 19581 webserver.cc:460] Webserver started at http://127.12.138.193:36283/ using document root <none> and password file <none>
I20260811 02:52:18.581629 19581 fs_manager.cc:359] Metadata directory not provided
I20260811 02:52:18.581696 19581 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 02:52:18.581954 19581 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 02:52:18.584101 19581 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786416707705494-12843-0/minicluster-data/ts-0/data/instance:
uuid: "b417085366db47abbda3f99b24bfac50"
format_stamp: "Formatted at 2026-08-11 02:52:18 on dist-test-slave-9gcw"
server_key: "467acc0a75ec577748ee48bc7c985ecc"
server_key_iv: "aa38ce423bf0fd1bce457423ba861d83"
server_key_version: "encryptionkey@0"
I20260811 02:52:18.584728 19581 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786416707705494-12843-0/minicluster-data/ts-0/wal/instance:
uuid: "b417085366db47abbda3f99b24bfac50"
format_stamp: "Formatted at 2026-08-11 02:52:18 on dist-test-slave-9gcw"
server_key: "467acc0a75ec577748ee48bc7c985ecc"
server_key_iv: "aa38ce423bf0fd1bce457423ba861d83"
server_key_version: "encryptionkey@0"
I20260811 02:52:18.588578 19581 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260811 02:52:18.591110 19601 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 02:52:18.592319 19581 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260811 02:52:18.592461 19581 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786416707705494-12843-0/minicluster-data/ts-0/data,/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786416707705494-12843-0/minicluster-data/ts-0/wal
uuid: "b417085366db47abbda3f99b24bfac50"
format_stamp: "Formatted at 2026-08-11 02:52:18 on dist-test-slave-9gcw"
server_key: "467acc0a75ec577748ee48bc7c985ecc"
server_key_iv: "aa38ce423bf0fd1bce457423ba861d83"
server_key_version: "encryptionkey@0"
I20260811 02:52:18.592573 19581 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786416707705494-12843-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786416707705494-12843-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786416707705494-12843-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 02:52:18.615814 19581 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 02:52:18.618991 19581 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 02:52:18.619211 19581 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 02:52:18.619843 19581 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 02:52:18.620721 19581 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 02:52:18.620806 19581 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 02:52:18.620903 19581 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 02:52:18.620956 19581 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 02:52:18.630209 19581 rpc_server.cc:307] RPC server started. Bound to: 127.12.138.193:39767
I20260811 02:52:18.630232 19734 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.138.193:39767 every 8 connection(s)
I20260811 02:52:18.631219 19581 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786416707705494-12843-0/minicluster-data/ts-0/data/info.pb
I20260811 02:52:18.634452 12843 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDawEwS/build/debug/bin/kudu as pid 19581
I20260811 02:52:18.634608 12843 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786416707705494-12843-0/minicluster-data/ts-0/wal/instance
I20260811 02:52:18.634907 12843 external_mini_cluster.cc:1428] Setting key 6c50e6205fc67d5d62c4629656b274e6
Aug 11 02:52:18 dist-test-slave-9gcw krb5kdc[19457](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416738, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.138.193@KRBTEST.COM for kudu/127.12.138.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.12.138.194@KRBTEST.COM; defaulting to no policy
I20260811 02:52:18.648213 19555 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:52:18.633206 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.138.193:40535 (local address 127.12.138.254:38913)
0811 02:52:18.633362 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0811 02:52:18.633368 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 02:52:18.634370 (+ 1002us) server_negotiation.cc:407] Connection header received
0811 02:52:18.636309 (+ 1939us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:52:18.636313 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:52:18.636374 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:52:18.636496 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:52:18.638412 (+ 1916us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:52:18.639206 (+ 794us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:52:18.640542 (+ 1336us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:52:18.641211 (+ 669us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:52:18.641348 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:52:18.644292 (+ 2944us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:52:18.644299 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:52:18.644304 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:52:18.644341 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:52:18.646095 (+ 1754us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:52:18.646622 (+ 527us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:52:18.646628 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:52:18.646630 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:52:18.646724 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:52:18.647081 (+ 357us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:52:18.647084 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:52:18.647086 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:52:18.647372 (+ 286us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:52:18.647490 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0811 02:52:18.647952 (+ 462us) server_negotiation.cc:299] Negotiation successful
0811 02:52:18.648076 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260811 02:52:18.649080 19737 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:52:18.633623 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.138.254:38913 (local address 127.12.138.193:40535)
0811 02:52:18.634169 (+ 546us) negotiation.cc:107] Waiting for socket to connect
0811 02:52:18.634225 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0811 02:52:18.636059 (+ 1834us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:52:18.636708 (+ 649us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:52:18.636716 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:52:18.637395 (+ 679us) client_negotiation.cc:189] Negotiated authn=SASL
0811 02:52:18.638204 (+ 809us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:52:18.638230 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:52:18.639373 (+ 1143us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:52:18.639376 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:52:18.640397 (+ 1021us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:52:18.640407 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:52:18.641381 (+ 974us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:52:18.641384 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:52:18.641542 (+ 158us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:52:18.642197 (+ 655us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 02:52:18.642215 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0811 02:52:18.644103 (+ 1888us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 02:52:18.646227 (+ 2124us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:52:18.646232 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:52:18.646244 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:52:18.646514 (+ 270us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:52:18.646833 (+ 319us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:52:18.646836 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:52:18.646837 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:52:18.646984 (+ 147us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:52:18.647479 (+ 495us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 02:52:18.647484 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 02:52:18.647722 (+ 238us) client_negotiation.cc:769] Sending connection context
0811 02:52:18.647887 (+ 165us) client_negotiation.cc:240] Negotiation successful
0811 02:52:18.648148 (+ 261us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":336,"thread_start_us":123,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.138.194@KRBTEST.COM" created.
I20260811 02:52:18.650483 19735 heartbeater.cc:347] Connected to a master server at 127.12.138.254:38913
I20260811 02:52:18.650738 19735 heartbeater.cc:464] Registering TS with master...
I20260811 02:52:18.651372 19735 heartbeater.cc:511] Master 127.12.138.254:38913 requested a full tablet report, sending...
I20260811 02:52:18.653226 19509 ts_manager.cc:194] Registered new tserver with Master: b417085366db47abbda3f99b24bfac50 (127.12.138.193:39767)
I20260811 02:52:18.654983 19509 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.12.138.193@KRBTEST.COM'} at 127.12.138.193:40535
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.138.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786416707705494-12843-0/krb5kdc/kudu_127.12.138.194.keytab.
Entry for principal kudu/127.12.138.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786416707705494-12843-0/krb5kdc/kudu_127.12.138.194.keytab.
I20260811 02:52:18.669329 12843 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDawEwS/build/debug/bin/kudu
/tmp/dist-test-taskDawEwS/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786416707705494-12843-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786416707705494-12843-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786416707705494-12843-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786416707705494-12843-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.138.194:0
--local_ip_for_outbound_sockets=127.12.138.194
--webserver_interface=127.12.138.194
--webserver_port=0
--tserver_master_addrs=127.12.138.254:38913
--keytab_file=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786416707705494-12843-0/krb5kdc/kudu_127.12.138.194.keytab
--principal=kudu/127.12.138.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.138.212:41549
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786416707705494-12843-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786416707705494-12843-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786416707705494-12843-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 02:52:18.789270 19741 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 02:52:18.789561 19741 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 02:52:18.789606 19741 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 02:52:18.793661 19741 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 02:52:18.793731 19741 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 02:52:18.793792 19741 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.138.194
I20260811 02:52:18.799243 19741 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.138.212:41549
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786416707705494-12843-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786416707705494-12843-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786416707705494-12843-0/krb5kdc/kudu_127.12.138.194.keytab
--principal=kudu/127.12.138.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.138.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786416707705494-12843-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.12.138.194
--webserver_port=0
--tserver_master_addrs=127.12.138.254:38913
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19741
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.138.194
--log_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786416707705494-12843-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 02:09:49 UTC on ccycloud.quasar-fsnqqc.root.comops.site
build id 4579570
I20260811 02:52:18.800294 19741 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 02:52:18.801306 19741 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 02:52:18.811841 19751 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:52:18.811846 19747 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:52:18.811846 19749 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:52:18.812000 19741 server_base.cc:1034] running on GCE node
I20260811 02:52:18.812541 19741 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 02:52:18.813274 19741 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 02:52:18.814455 19741 hybrid_clock.cc:648] HybridClock initialized: now 1786416738814438 us; error 27 us; skew 500 ppm
Aug 11 02:52:18 dist-test-slave-9gcw krb5kdc[19457](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416738, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.138.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 02:52:18.817628 19741 init.cc:377] Logged in from keytab as kudu/127.12.138.194@KRBTEST.COM (short username kudu)
I20260811 02:52:18.818580 19741 webserver.cc:460] Webserver started at http://127.12.138.194:32903/ using document root <none> and password file <none>
I20260811 02:52:18.819167 19741 fs_manager.cc:359] Metadata directory not provided
I20260811 02:52:18.819238 19741 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 02:52:18.819423 19741 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 02:52:18.821614 19741 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786416707705494-12843-0/minicluster-data/ts-1/data/instance:
uuid: "b0ba0674021d4918b74c7af05909eade"
format_stamp: "Formatted at 2026-08-11 02:52:18 on dist-test-slave-9gcw"
server_key: "4a9460a4a2be85aacc3ab989835e3cf9"
server_key_iv: "7c529d64dfe43c4b10b7422d6828b9dd"
server_key_version: "encryptionkey@0"
I20260811 02:52:18.822191 19741 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786416707705494-12843-0/minicluster-data/ts-1/wal/instance:
uuid: "b0ba0674021d4918b74c7af05909eade"
format_stamp: "Formatted at 2026-08-11 02:52:18 on dist-test-slave-9gcw"
server_key: "4a9460a4a2be85aacc3ab989835e3cf9"
server_key_iv: "7c529d64dfe43c4b10b7422d6828b9dd"
server_key_version: "encryptionkey@0"
I20260811 02:52:18.826134 19741 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260811 02:52:18.829058 19761 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 02:52:18.830469 19741 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260811 02:52:18.830626 19741 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786416707705494-12843-0/minicluster-data/ts-1/data,/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786416707705494-12843-0/minicluster-data/ts-1/wal
uuid: "b0ba0674021d4918b74c7af05909eade"
format_stamp: "Formatted at 2026-08-11 02:52:18 on dist-test-slave-9gcw"
server_key: "4a9460a4a2be85aacc3ab989835e3cf9"
server_key_iv: "7c529d64dfe43c4b10b7422d6828b9dd"
server_key_version: "encryptionkey@0"
I20260811 02:52:18.830753 19741 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786416707705494-12843-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786416707705494-12843-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786416707705494-12843-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 02:52:18.846884 19741 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 02:52:18.850256 19741 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 02:52:18.850505 19741 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 02:52:18.851248 19741 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 02:52:18.852116 19741 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 02:52:18.852193 19741 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 02:52:18.852267 19741 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 02:52:18.852320 19741 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 02:52:18.862707 19741 rpc_server.cc:307] RPC server started. Bound to: 127.12.138.194:44011
I20260811 02:52:18.862727 19892 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.138.194:44011 every 8 connection(s)
I20260811 02:52:18.863740 19741 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786416707705494-12843-0/minicluster-data/ts-1/data/info.pb
I20260811 02:52:18.867195 12843 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDawEwS/build/debug/bin/kudu as pid 19741
I20260811 02:52:18.867326 12843 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786416707705494-12843-0/minicluster-data/ts-1/wal/instance
I20260811 02:52:18.867779 12843 external_mini_cluster.cc:1428] Setting key 60be4a8e8894af80e61093a3a97416d3
Aug 11 02:52:18 dist-test-slave-9gcw krb5kdc[19457](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416738, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.138.194@KRBTEST.COM for kudu/127.12.138.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.12.138.195@KRBTEST.COM; defaulting to no policy
I20260811 02:52:18.880488 19555 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:52:18.865720 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.138.194:46545 (local address 127.12.138.254:38913)
0811 02:52:18.865874 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0811 02:52:18.865878 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 02:52:18.866839 (+ 961us) server_negotiation.cc:407] Connection header received
0811 02:52:18.868816 (+ 1977us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:52:18.868820 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:52:18.868875 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:52:18.868989 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:52:18.870656 (+ 1667us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:52:18.871410 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:52:18.872857 (+ 1447us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:52:18.873098 (+ 241us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:52:18.873281 (+ 183us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:52:18.876275 (+ 2994us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:52:18.876278 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:52:18.876280 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:52:18.876307 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:52:18.878090 (+ 1783us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:52:18.878740 (+ 650us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:52:18.878748 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:52:18.878749 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:52:18.878836 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:52:18.879208 (+ 372us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:52:18.879213 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:52:18.879215 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:52:18.879468 (+ 253us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:52:18.879583 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0811 02:52:18.880150 (+ 567us) server_negotiation.cc:299] Negotiation successful
0811 02:52:18.880291 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260811 02:52:18.881418 19895 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:52:18.866145 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.138.254:38913 (local address 127.12.138.194:46545)
0811 02:52:18.866636 (+ 491us) negotiation.cc:107] Waiting for socket to connect
0811 02:52:18.866712 (+ 76us) client_negotiation.cc:174] Beginning negotiation
0811 02:52:18.868585 (+ 1873us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:52:18.869235 (+ 650us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:52:18.869244 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:52:18.869701 (+ 457us) client_negotiation.cc:189] Negotiated authn=SASL
0811 02:52:18.870440 (+ 739us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:52:18.870465 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:52:18.871559 (+ 1094us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:52:18.871565 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:52:18.872709 (+ 1144us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:52:18.872719 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:52:18.873323 (+ 604us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:52:18.873328 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:52:18.873515 (+ 187us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:52:18.874234 (+ 719us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 02:52:18.874252 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0811 02:52:18.876065 (+ 1813us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 02:52:18.878290 (+ 2225us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:52:18.878297 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:52:18.878314 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:52:18.878614 (+ 300us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:52:18.878938 (+ 324us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:52:18.878941 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:52:18.878943 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:52:18.879105 (+ 162us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:52:18.879582 (+ 477us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 02:52:18.879590 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 02:52:18.879878 (+ 288us) client_negotiation.cc:769] Sending connection context
0811 02:52:18.880081 (+ 203us) client_negotiation.cc:240] Negotiation successful
0811 02:52:18.880449 (+ 368us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":277,"thread_start_us":106,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.138.195@KRBTEST.COM" created.
I20260811 02:52:18.883068 19893 heartbeater.cc:347] Connected to a master server at 127.12.138.254:38913
I20260811 02:52:18.883355 19893 heartbeater.cc:464] Registering TS with master...
I20260811 02:52:18.883949 19893 heartbeater.cc:511] Master 127.12.138.254:38913 requested a full tablet report, sending...
I20260811 02:52:18.885270 19509 ts_manager.cc:194] Registered new tserver with Master: b0ba0674021d4918b74c7af05909eade (127.12.138.194:44011)
I20260811 02:52:18.885916 19509 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.12.138.194@KRBTEST.COM'} at 127.12.138.194:46545
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.138.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786416707705494-12843-0/krb5kdc/kudu_127.12.138.195.keytab.
Entry for principal kudu/127.12.138.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786416707705494-12843-0/krb5kdc/kudu_127.12.138.195.keytab.
I20260811 02:52:18.901912 12843 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDawEwS/build/debug/bin/kudu
/tmp/dist-test-taskDawEwS/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786416707705494-12843-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786416707705494-12843-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786416707705494-12843-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786416707705494-12843-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.138.195:0
--local_ip_for_outbound_sockets=127.12.138.195
--webserver_interface=127.12.138.195
--webserver_port=0
--tserver_master_addrs=127.12.138.254:38913
--keytab_file=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786416707705494-12843-0/krb5kdc/kudu_127.12.138.195.keytab
--principal=kudu/127.12.138.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.138.212:41549
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786416707705494-12843-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786416707705494-12843-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786416707705494-12843-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 02:52:19.034653 19898 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 02:52:19.035008 19898 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 02:52:19.035094 19898 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 02:52:19.039417 19898 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 02:52:19.039492 19898 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 02:52:19.039554 19898 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.138.195
I20260811 02:52:19.045403 19898 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.138.212:41549
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786416707705494-12843-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786416707705494-12843-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786416707705494-12843-0/krb5kdc/kudu_127.12.138.195.keytab
--principal=kudu/127.12.138.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.138.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786416707705494-12843-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.12.138.195
--webserver_port=0
--tserver_master_addrs=127.12.138.254:38913
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19898
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.138.195
--log_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786416707705494-12843-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 02:09:49 UTC on ccycloud.quasar-fsnqqc.root.comops.site
build id 4579570
I20260811 02:52:19.046509 19898 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 02:52:19.047510 19898 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260811 02:52:19.057745 19898 server_base.cc:1034] running on GCE node
W20260811 02:52:19.057708 19907 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:52:19.057732 19904 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:52:19.057749 19905 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:52:19.058509 19898 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 02:52:19.059190 19898 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 02:52:19.060359 19898 hybrid_clock.cc:648] HybridClock initialized: now 1786416739060335 us; error 41 us; skew 500 ppm
Aug 11 02:52:19 dist-test-slave-9gcw krb5kdc[19457](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416739, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.138.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 02:52:19.063616 19898 init.cc:377] Logged in from keytab as kudu/127.12.138.195@KRBTEST.COM (short username kudu)
I20260811 02:52:19.064535 19898 webserver.cc:460] Webserver started at http://127.12.138.195:43793/ using document root <none> and password file <none>
I20260811 02:52:19.065224 19898 fs_manager.cc:359] Metadata directory not provided
I20260811 02:52:19.065320 19898 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 02:52:19.065551 19898 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 02:52:19.067638 19898 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786416707705494-12843-0/minicluster-data/ts-2/data/instance:
uuid: "0e77c0b1382649f3b89c177570dded38"
format_stamp: "Formatted at 2026-08-11 02:52:19 on dist-test-slave-9gcw"
server_key: "a943e171e6212331155c762edd637fe0"
server_key_iv: "c50724a3887d64ebad21577177c9673d"
server_key_version: "encryptionkey@0"
I20260811 02:52:19.068225 19898 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786416707705494-12843-0/minicluster-data/ts-2/wal/instance:
uuid: "0e77c0b1382649f3b89c177570dded38"
format_stamp: "Formatted at 2026-08-11 02:52:19 on dist-test-slave-9gcw"
server_key: "a943e171e6212331155c762edd637fe0"
server_key_iv: "c50724a3887d64ebad21577177c9673d"
server_key_version: "encryptionkey@0"
I20260811 02:52:19.072111 19898 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260811 02:52:19.074635 19917 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 02:52:19.075902 19898 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260811 02:52:19.076047 19898 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786416707705494-12843-0/minicluster-data/ts-2/data,/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786416707705494-12843-0/minicluster-data/ts-2/wal
uuid: "0e77c0b1382649f3b89c177570dded38"
format_stamp: "Formatted at 2026-08-11 02:52:19 on dist-test-slave-9gcw"
server_key: "a943e171e6212331155c762edd637fe0"
server_key_iv: "c50724a3887d64ebad21577177c9673d"
server_key_version: "encryptionkey@0"
I20260811 02:52:19.076164 19898 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786416707705494-12843-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786416707705494-12843-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786416707705494-12843-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 02:52:19.104283 19898 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 02:52:19.108472 19898 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 02:52:19.108752 19898 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 02:52:19.109520 19898 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 02:52:19.110507 19898 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 02:52:19.110565 19898 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 02:52:19.110675 19898 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 02:52:19.110724 19898 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 02:52:19.121861 19898 rpc_server.cc:307] RPC server started. Bound to: 127.12.138.195:45303
I20260811 02:52:19.121953 20052 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.138.195:45303 every 8 connection(s)
I20260811 02:52:19.122970 19898 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786416707705494-12843-0/minicluster-data/ts-2/data/info.pb
I20260811 02:52:19.130860 12843 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDawEwS/build/debug/bin/kudu as pid 19898
I20260811 02:52:19.131022 12843 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786416707705494-12843-0/minicluster-data/ts-2/wal/instance
I20260811 02:52:19.131397 12843 external_mini_cluster.cc:1428] Setting key 8369cb5bcc0b091b3f765c04f74955ca
Aug 11 02:52:19 dist-test-slave-9gcw krb5kdc[19457](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416739, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.138.195@KRBTEST.COM for kudu/127.12.138.254@KRBTEST.COM
I20260811 02:52:19.139133 19555 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:52:19.125127 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.138.195:44101 (local address 127.12.138.254:38913)
0811 02:52:19.125325 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0811 02:52:19.125330 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 02:52:19.126236 (+ 906us) server_negotiation.cc:407] Connection header received
0811 02:52:19.127678 (+ 1442us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:52:19.127681 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:52:19.127728 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:52:19.127826 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:52:19.129381 (+ 1555us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:52:19.129909 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:52:19.131755 (+ 1846us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:52:19.132065 (+ 310us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:52:19.132182 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:52:19.135610 (+ 3428us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:52:19.135613 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:52:19.135616 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:52:19.135640 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:52:19.137044 (+ 1404us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:52:19.137612 (+ 568us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:52:19.137615 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:52:19.137616 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:52:19.137692 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:52:19.138068 (+ 376us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:52:19.138070 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:52:19.138071 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:52:19.138250 (+ 179us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:52:19.138329 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0811 02:52:19.138907 (+ 578us) server_negotiation.cc:299] Negotiation successful
0811 02:52:19.139010 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260811 02:52:19.139983 20055 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:52:19.125601 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.138.254:38913 (local address 127.12.138.195:44101)
0811 02:52:19.126073 (+ 472us) negotiation.cc:107] Waiting for socket to connect
0811 02:52:19.126115 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0811 02:52:19.127477 (+ 1362us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:52:19.128034 (+ 557us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:52:19.128042 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:52:19.128459 (+ 417us) client_negotiation.cc:189] Negotiated authn=SASL
0811 02:52:19.129130 (+ 671us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:52:19.129221 (+ 91us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:52:19.130081 (+ 860us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:52:19.130084 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:52:19.131589 (+ 1505us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:52:19.131603 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:52:19.132277 (+ 674us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:52:19.132284 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:52:19.132549 (+ 265us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:52:19.133406 (+ 857us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 02:52:19.133424 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0811 02:52:19.135454 (+ 2030us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 02:52:19.137217 (+ 1763us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:52:19.137222 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:52:19.137237 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:52:19.137512 (+ 275us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:52:19.137800 (+ 288us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:52:19.137803 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:52:19.137804 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:52:19.137966 (+ 162us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:52:19.138349 (+ 383us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 02:52:19.138355 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 02:52:19.138609 (+ 254us) client_negotiation.cc:769] Sending connection context
0811 02:52:19.138868 (+ 259us) client_negotiation.cc:240] Negotiation successful
0811 02:52:19.139134 (+ 266us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":281,"thread_start_us":120,"threads_started":1}
I20260811 02:52:19.141211 20053 heartbeater.cc:347] Connected to a master server at 127.12.138.254:38913
I20260811 02:52:19.141481 20053 heartbeater.cc:464] Registering TS with master...
I20260811 02:52:19.142051 20053 heartbeater.cc:511] Master 127.12.138.254:38913 requested a full tablet report, sending...
I20260811 02:52:19.143107 19509 ts_manager.cc:194] Registered new tserver with Master: 0e77c0b1382649f3b89c177570dded38 (127.12.138.195:45303)
I20260811 02:52:19.143704 19509 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.12.138.195@KRBTEST.COM'} at 127.12.138.195:44101
I20260811 02:52:19.145733 12843 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal test-admin with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786416707705494-12843-0/krb5kdc/test-admin.keytab.
Entry for principal test-admin with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786416707705494-12843-0/krb5kdc/test-admin.keytab.
Aug 11 02:52:19 dist-test-slave-9gcw krb5kdc[19457](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416739, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 02:52:19.166602 12843 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
W20260811 02:52:19.169478 20061 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 02:52:19.168004 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52664 (local address 127.12.138.193:39767)
0811 02:52:19.168362 (+ 358us) server_negotiation.cc:206] Beginning negotiation
0811 02:52:19.168369 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 02:52:19.168394 (+ 25us) server_negotiation.cc:407] Connection header received
0811 02:52:19.168480 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:52:19.168486 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:52:19.168671 (+ 185us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:52:19.168792 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:52:19.169277 (+ 485us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:52664: BlockingRecv error: recv got EOF from 127.0.0.1:52664 (error 108)
Metrics: {"server-negotiator.queue_time_us":231,"thread_start_us":135,"threads_started":1}
W20260811 02:52:19.169771 12843 init.cc:244] Error while opening credential cache '/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786416707705494-12843-0/krb5kdc/krb5cc' of type FILE: Runtime error: End of credential cache reached (filename: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786416707705494-12843-0/krb5kdc/krb5cc)
Aug 11 02:52:19 dist-test-slave-9gcw krb5kdc[19457](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416739, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 02:52:19.170879 12843 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 11 02:52:19 dist-test-slave-9gcw krb5kdc[19457](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416739, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 02:52:19.171922 12843 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 11 02:52:19 dist-test-slave-9gcw krb5kdc[19457](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416739, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.12.138.193@KRBTEST.COM
I20260811 02:52:19.182330 20061 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:52:19.172954 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52676 (local address 127.12.138.193:39767)
0811 02:52:19.173082 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0811 02:52:19.173085 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 02:52:19.173212 (+ 127us) server_negotiation.cc:407] Connection header received
0811 02:52:19.173348 (+ 136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:52:19.173350 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:52:19.173410 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:52:19.173480 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:52:19.174165 (+ 685us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:52:19.174907 (+ 742us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:52:19.175824 (+ 917us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:52:19.176178 (+ 354us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:52:19.176257 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:52:19.178582 (+ 2325us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:52:19.178597 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:52:19.178610 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:52:19.178633 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:52:19.180387 (+ 1754us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:52:19.180849 (+ 462us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:52:19.180854 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:52:19.180856 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:52:19.180973 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:52:19.181326 (+ 353us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:52:19.181329 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:52:19.181331 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:52:19.181574 (+ 243us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:52:19.181785 (+ 211us) server_negotiation.cc:1035] Waiting for connection context
0811 02:52:19.182043 (+ 258us) server_negotiation.cc:299] Negotiation successful
0811 02:52:19.182197 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
W20260811 02:52:19.186215 20061 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 02:52:19.185101 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52690 (local address 127.12.138.193:39767)
0811 02:52:19.185244 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0811 02:52:19.185249 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 02:52:19.185392 (+ 143us) server_negotiation.cc:407] Connection header received
0811 02:52:19.185528 (+ 136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:52:19.185557 (+ 29us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:52:19.185594 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:52:19.185684 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:52:19.186102 (+ 418us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:52690: BlockingRecv error: recv got EOF from 127.0.0.1:52690 (error 108)
Metrics: {"server-negotiator.queue_time_us":30}
W20260811 02:52:19.186422 12843 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786416707705494-12843-0/krb5kdc/krb5cc' of type FILE
Aug 11 02:52:19 dist-test-slave-9gcw krb5kdc[19457](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416739, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 02:52:19.187479 12843 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 11 02:52:19 dist-test-slave-9gcw krb5kdc[19457](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416739, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 02:52:19.188367 12843 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 11 02:52:19 dist-test-slave-9gcw krb5kdc[19457](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416739, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.12.138.193@KRBTEST.COM
I20260811 02:52:19.197245 20061 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:52:19.189270 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52704 (local address 127.12.138.193:39767)
0811 02:52:19.189389 (+ 119us) server_negotiation.cc:206] Beginning negotiation
0811 02:52:19.189396 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 02:52:19.189431 (+ 35us) server_negotiation.cc:407] Connection header received
0811 02:52:19.189583 (+ 152us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:52:19.189586 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:52:19.189635 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:52:19.189727 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:52:19.190248 (+ 521us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:52:19.190769 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:52:19.191564 (+ 795us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:52:19.191828 (+ 264us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:52:19.191942 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:52:19.194127 (+ 2185us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:52:19.194136 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:52:19.194138 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:52:19.194161 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:52:19.195392 (+ 1231us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:52:19.195807 (+ 415us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:52:19.195814 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:52:19.195819 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:52:19.195909 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:52:19.196274 (+ 365us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:52:19.196279 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:52:19.196281 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:52:19.196475 (+ 194us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:52:19.196626 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0811 02:52:19.196880 (+ 254us) server_negotiation.cc:299] Negotiation successful
0811 02:52:19.197004 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
W20260811 02:52:19.200349 20061 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 02:52:19.199162 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52718 (local address 127.12.138.193:39767)
0811 02:52:19.199315 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0811 02:52:19.199319 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 02:52:19.199444 (+ 125us) server_negotiation.cc:407] Connection header received
0811 02:52:19.199612 (+ 168us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:52:19.199615 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:52:19.199727 (+ 112us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:52:19.199828 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:52:19.200236 (+ 408us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:52718: BlockingRecv error: recv got EOF from 127.0.0.1:52718 (error 108)
Metrics: {"server-negotiator.queue_time_us":69}
W20260811 02:52:19.200518 12843 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786416707705494-12843-0/krb5kdc/krb5cc' of type FILE
Aug 11 02:52:19 dist-test-slave-9gcw krb5kdc[19457](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416739, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 02:52:19.201607 12843 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 11 02:52:19 dist-test-slave-9gcw krb5kdc[19457](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416739, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 02:52:19.202484 12843 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 11 02:52:19 dist-test-slave-9gcw krb5kdc[19457](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416739, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.12.138.193@KRBTEST.COM
I20260811 02:52:19.210968 20061 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:52:19.203262 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52732 (local address 127.12.138.193:39767)
0811 02:52:19.203382 (+ 120us) server_negotiation.cc:206] Beginning negotiation
0811 02:52:19.203386 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 02:52:19.203520 (+ 134us) server_negotiation.cc:407] Connection header received
0811 02:52:19.203600 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:52:19.203603 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:52:19.203636 (+ 33us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:52:19.203704 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:52:19.204158 (+ 454us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:52:19.204668 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:52:19.205534 (+ 866us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:52:19.205763 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:52:19.205848 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:52:19.208074 (+ 2226us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:52:19.208078 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:52:19.208080 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:52:19.208111 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:52:19.209536 (+ 1425us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:52:19.209920 (+ 384us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:52:19.209923 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:52:19.209924 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:52:19.209999 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:52:19.210226 (+ 227us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:52:19.210228 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:52:19.210229 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:52:19.210393 (+ 164us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:52:19.210507 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0811 02:52:19.210654 (+ 147us) server_negotiation.cc:299] Negotiation successful
0811 02:52:19.210747 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260811 02:52:19.212016 12843 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDawEwS/build/debug/bin/kudu with pid 19581
I20260811 02:52:19.222178 12843 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDawEwS/build/debug/bin/kudu with pid 19741
I20260811 02:52:19.235143 12843 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDawEwS/build/debug/bin/kudu with pid 19898
I20260811 02:52:19.244203 12843 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDawEwS/build/debug/bin/kudu with pid 19475
2026-08-11T02:52:19Z chronyd exiting
[ OK ] SecurityITest.TestCorruptKerberosCC (2398 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipalMultipleMaster
Loading random data
Initializing database '/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786416707705494-12843-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 02:52:19 dist-test-slave-9gcw krb5kdc[20077](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 02:52:19 dist-test-slave-9gcw krb5kdc[20077](info): set up 2 sockets
Aug 11 02:52:19 dist-test-slave-9gcw krb5kdc[20077](info): commencing operation
krb5kdc: starting...
W20260811 02:52:20.400844 12843 mini_kdc.cc:121] Time spent starting KDC: real 1.131s user 0.002s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786416707705494-12843-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786416707705494-12843-0/krb5kdc/test-user.keytab.
Aug 11 02:52:20 dist-test-slave-9gcw krb5kdc[20077](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416740, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T02:52:20Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T02:52:20Z Disabled control of system clock
WARNING: no policy specified for oryx/127.12.138.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.12.138.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.12.138.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786416707705494-12843-0/krb5kdc/oryx_127.12.138.254.keytab.
Entry for principal oryx/127.12.138.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786416707705494-12843-0/krb5kdc/oryx_127.12.138.254.keytab.
I20260811 02:52:20.536914 12843 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDawEwS/build/debug/bin/kudu
/tmp/dist-test-taskDawEwS/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786416707705494-12843-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786416707705494-12843-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786416707705494-12843-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786416707705494-12843-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.138.254:43457
--webserver_interface=127.12.138.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786416707705494-12843-0/krb5kdc/oryx_127.12.138.254.keytab
--principal=oryx/127.12.138.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.138.212:41877
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.12.138.254:43457
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786416707705494-12843-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786416707705494-12843-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786416707705494-12843-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 02:52:20.655020 20094 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 02:52:20.655453 20094 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 02:52:20.655577 20094 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 02:52:20.660318 20094 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 02:52:20.660440 20094 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 02:52:20.660485 20094 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 02:52:20.660529 20094 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 02:52:20.660568 20094 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 02:52:20.666790 20094 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.138.212:41877
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786416707705494-12843-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786416707705494-12843-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.12.138.254:43457
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786416707705494-12843-0/krb5kdc/oryx_127.12.138.254.keytab
--principal=oryx/127.12.138.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.138.254:43457
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786416707705494-12843-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.12.138.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20094
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786416707705494-12843-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 02:09:49 UTC on ccycloud.quasar-fsnqqc.root.comops.site
build id 4579570
I20260811 02:52:20.668069 20094 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 02:52:20.669245 20094 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 02:52:20.678543 20104 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:52:20.678565 20101 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:52:20.678565 20102 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:52:20.678800 20094 server_base.cc:1034] running on GCE node
I20260811 02:52:20.679540 20094 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 02:52:20.680707 20094 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 02:52:20.681941 20094 hybrid_clock.cc:648] HybridClock initialized: now 1786416740681940 us; error 49 us; skew 500 ppm
Aug 11 02:52:20 dist-test-slave-9gcw krb5kdc[20077](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416740, etypes {rep=17 tkt=17 ses=17}, oryx/127.12.138.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 02:52:20.684955 20094 init.cc:377] Logged in from keytab as oryx/127.12.138.254@KRBTEST.COM (short username oryx)
I20260811 02:52:20.685961 20094 webserver.cc:460] Webserver started at http://127.12.138.254:40669/ using document root <none> and password file <none>
I20260811 02:52:20.686507 20094 fs_manager.cc:359] Metadata directory not provided
I20260811 02:52:20.686569 20094 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 02:52:20.686740 20094 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 02:52:20.688838 20094 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786416707705494-12843-0/minicluster-data/master-0/data/instance:
uuid: "0317e693c61849e9bf42d55fbf6ef321"
format_stamp: "Formatted at 2026-08-11 02:52:20 on dist-test-slave-9gcw"
server_key: "d198608498609e947087f90443d44f3f"
server_key_iv: "8b32fc8b65aab95e91d5811bd781a273"
server_key_version: "encryptionkey@0"
I20260811 02:52:20.689478 20094 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786416707705494-12843-0/minicluster-data/master-0/wal/instance:
uuid: "0317e693c61849e9bf42d55fbf6ef321"
format_stamp: "Formatted at 2026-08-11 02:52:20 on dist-test-slave-9gcw"
server_key: "d198608498609e947087f90443d44f3f"
server_key_iv: "8b32fc8b65aab95e91d5811bd781a273"
server_key_version: "encryptionkey@0"
I20260811 02:52:20.693377 20094 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260811 02:52:20.695844 20112 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 02:52:20.697079 20094 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260811 02:52:20.697273 20094 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786416707705494-12843-0/minicluster-data/master-0/data,/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786416707705494-12843-0/minicluster-data/master-0/wal
uuid: "0317e693c61849e9bf42d55fbf6ef321"
format_stamp: "Formatted at 2026-08-11 02:52:20 on dist-test-slave-9gcw"
server_key: "d198608498609e947087f90443d44f3f"
server_key_iv: "8b32fc8b65aab95e91d5811bd781a273"
server_key_version: "encryptionkey@0"
I20260811 02:52:20.697376 20094 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786416707705494-12843-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786416707705494-12843-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786416707705494-12843-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 02:52:20.707715 20094 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 02:52:20.710829 20094 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 02:52:20.710997 20094 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 02:52:20.718801 20094 rpc_server.cc:307] RPC server started. Bound to: 127.12.138.254:43457
I20260811 02:52:20.719133 20176 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.138.254:43457 every 8 connection(s)
I20260811 02:52:20.719854 20094 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786416707705494-12843-0/minicluster-data/master-0/data/info.pb
I20260811 02:52:20.722798 20177 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 02:52:20.724059 12843 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDawEwS/build/debug/bin/kudu as pid 20094
I20260811 02:52:20.724215 12843 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786416707705494-12843-0/minicluster-data/master-0/wal/instance
I20260811 02:52:20.724566 12843 external_mini_cluster.cc:1428] Setting key fbb24aaeb24ab4be5aadd32e69fe6515
I20260811 02:52:20.729514 20177 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0317e693c61849e9bf42d55fbf6ef321: Bootstrap starting.
I20260811 02:52:20.732276 20177 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0317e693c61849e9bf42d55fbf6ef321: Neither blocks nor log segments found. Creating new log.
I20260811 02:52:20.733287 20177 log.cc:824] T 00000000000000000000000000000000 P 0317e693c61849e9bf42d55fbf6ef321: Log is configured to *not* fsync() on all Append() calls
Aug 11 02:52:20 dist-test-slave-9gcw krb5kdc[20077](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416740, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.12.138.254@KRBTEST.COM
I20260811 02:52:20.735524 20177 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0317e693c61849e9bf42d55fbf6ef321: No bootstrap required, opened a new log
I20260811 02:52:20.739229 20182 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:52:20.726019 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54532 (local address 127.12.138.254:43457)
0811 02:52:20.726432 (+ 413us) server_negotiation.cc:206] Beginning negotiation
0811 02:52:20.726448 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0811 02:52:20.726485 (+ 37us) server_negotiation.cc:407] Connection header received
0811 02:52:20.727131 (+ 646us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:52:20.727148 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:52:20.727482 (+ 334us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:52:20.727761 (+ 279us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:52:20.728721 (+ 960us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:52:20.729855 (+ 1134us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:52:20.731014 (+ 1159us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:52:20.731477 (+ 463us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:52:20.731603 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:52:20.734490 (+ 2887us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:52:20.734495 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:52:20.734508 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:52:20.734535 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:52:20.736332 (+ 1797us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:52:20.736793 (+ 461us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:52:20.736804 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:52:20.736809 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:52:20.736920 (+ 111us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:52:20.737242 (+ 322us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:52:20.737247 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:52:20.737249 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:52:20.737648 (+ 399us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:52:20.737873 (+ 225us) server_negotiation.cc:1035] Waiting for connection context
0811 02:52:20.738156 (+ 283us) server_negotiation.cc:299] Negotiation successful
0811 02:52:20.738449 (+ 293us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":262,"thread_start_us":137,"threads_started":1}
I20260811 02:52:20.739244 20177 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0317e693c61849e9bf42d55fbf6ef321 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0317e693c61849e9bf42d55fbf6ef321" member_type: VOTER last_known_addr { host: "127.12.138.254" port: 43457 } }
I20260811 02:52:20.739550 20177 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0317e693c61849e9bf42d55fbf6ef321 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 02:52:20.739615 20177 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0317e693c61849e9bf42d55fbf6ef321 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0317e693c61849e9bf42d55fbf6ef321, State: Initialized, Role: FOLLOWER
I20260811 02:52:20.740013 20177 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0317e693c61849e9bf42d55fbf6ef321 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0317e693c61849e9bf42d55fbf6ef321" member_type: VOTER last_known_addr { host: "127.12.138.254" port: 43457 } }
I20260811 02:52:20.740121 20177 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0317e693c61849e9bf42d55fbf6ef321 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 02:52:20.740160 20177 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0317e693c61849e9bf42d55fbf6ef321 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 02:52:20.740224 20177 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0317e693c61849e9bf42d55fbf6ef321 [term 0 FOLLOWER]: Advancing to term 1
I20260811 02:52:20.741235 20177 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0317e693c61849e9bf42d55fbf6ef321 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0317e693c61849e9bf42d55fbf6ef321" member_type: VOTER last_known_addr { host: "127.12.138.254" port: 43457 } }
I20260811 02:52:20.741613 20177 leader_election.cc:302] T 00000000000000000000000000000000 P 0317e693c61849e9bf42d55fbf6ef321 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0317e693c61849e9bf42d55fbf6ef321; no voters:
I20260811 02:52:20.742003 20177 leader_election.cc:288] T 00000000000000000000000000000000 P 0317e693c61849e9bf42d55fbf6ef321 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 02:52:20.742118 20184 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0317e693c61849e9bf42d55fbf6ef321 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 02:52:20.742375 20184 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0317e693c61849e9bf42d55fbf6ef321 [term 1 LEADER]: Becoming Leader. State: Replica: 0317e693c61849e9bf42d55fbf6ef321, State: Running, Role: LEADER
I20260811 02:52:20.742679 20184 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0317e693c61849e9bf42d55fbf6ef321 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0317e693c61849e9bf42d55fbf6ef321" member_type: VOTER last_known_addr { host: "127.12.138.254" port: 43457 } }
I20260811 02:52:20.743257 20177 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0317e693c61849e9bf42d55fbf6ef321 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 02:52:20.744370 20186 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0317e693c61849e9bf42d55fbf6ef321 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0317e693c61849e9bf42d55fbf6ef321. Latest consensus state: current_term: 1 leader_uuid: "0317e693c61849e9bf42d55fbf6ef321" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0317e693c61849e9bf42d55fbf6ef321" member_type: VOTER last_known_addr { host: "127.12.138.254" port: 43457 } } }
I20260811 02:52:20.744509 20186 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0317e693c61849e9bf42d55fbf6ef321 [sys.catalog]: This master's current role is: LEADER
I20260811 02:52:20.744892 20193 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 02:52:20.746068 20185 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0317e693c61849e9bf42d55fbf6ef321 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0317e693c61849e9bf42d55fbf6ef321" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0317e693c61849e9bf42d55fbf6ef321" member_type: VOTER last_known_addr { host: "127.12.138.254" port: 43457 } } }
I20260811 02:52:20.746176 20185 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0317e693c61849e9bf42d55fbf6ef321 [sys.catalog]: This master's current role is: LEADER
I20260811 02:52:20.748056 20193 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 02:52:20.753809 20193 catalog_manager.cc:1347] Generated new cluster ID: 101bfd3e0cf345c7b289e94b98d49ad6
I20260811 02:52:20.753881 20193 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 02:52:20.771456 20193 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 02:52:20.772487 20193 catalog_manager.cc:1504] Loading token signing keys...
I20260811 02:52:20.781687 20193 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0317e693c61849e9bf42d55fbf6ef321: Generated new TSK 0
I20260811 02:52:20.782469 20193 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.12.138.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.12.138.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.12.138.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786416707705494-12843-0/krb5kdc/oryx_127.12.138.193.keytab.
Entry for principal oryx/127.12.138.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786416707705494-12843-0/krb5kdc/oryx_127.12.138.193.keytab.
I20260811 02:52:20.817128 12843 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDawEwS/build/debug/bin/kudu
/tmp/dist-test-taskDawEwS/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786416707705494-12843-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786416707705494-12843-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786416707705494-12843-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786416707705494-12843-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.138.193:0
--local_ip_for_outbound_sockets=127.12.138.193
--webserver_interface=127.12.138.193
--webserver_port=0
--tserver_master_addrs=127.12.138.254:43457
--keytab_file=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786416707705494-12843-0/krb5kdc/oryx_127.12.138.193.keytab
--principal=oryx/127.12.138.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.138.212:41877
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786416707705494-12843-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786416707705494-12843-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786416707705494-12843-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 02:52:20.939982 20207 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 02:52:20.940306 20207 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 02:52:20.940390 20207 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 02:52:20.945056 20207 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 02:52:20.945190 20207 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 02:52:20.945312 20207 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.138.193
I20260811 02:52:20.950728 20207 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.138.212:41877
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786416707705494-12843-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786416707705494-12843-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786416707705494-12843-0/krb5kdc/oryx_127.12.138.193.keytab
--principal=oryx/127.12.138.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.138.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786416707705494-12843-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.12.138.193
--webserver_port=0
--tserver_master_addrs=127.12.138.254:43457
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20207
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.138.193
--log_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786416707705494-12843-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 02:09:49 UTC on ccycloud.quasar-fsnqqc.root.comops.site
build id 4579570
I20260811 02:52:20.951890 20207 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 02:52:20.952972 20207 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 02:52:20.963131 20214 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:52:20.963248 20215 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:52:20.963136 20218 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:52:20.963490 20207 server_base.cc:1034] running on GCE node
I20260811 02:52:20.964012 20207 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 02:52:20.964674 20207 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 02:52:20.965859 20207 hybrid_clock.cc:648] HybridClock initialized: now 1786416740965842 us; error 30 us; skew 500 ppm
Aug 11 02:52:20 dist-test-slave-9gcw krb5kdc[20077](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416740, etypes {rep=17 tkt=17 ses=17}, oryx/127.12.138.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 02:52:20.968621 20207 init.cc:377] Logged in from keytab as oryx/127.12.138.193@KRBTEST.COM (short username oryx)
I20260811 02:52:20.969570 20207 webserver.cc:460] Webserver started at http://127.12.138.193:35107/ using document root <none> and password file <none>
I20260811 02:52:20.970144 20207 fs_manager.cc:359] Metadata directory not provided
I20260811 02:52:20.970238 20207 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 02:52:20.970489 20207 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 02:52:20.972534 20207 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786416707705494-12843-0/minicluster-data/ts-0/data/instance:
uuid: "ab096c76c5544f828e9eb7f2bee1977e"
format_stamp: "Formatted at 2026-08-11 02:52:20 on dist-test-slave-9gcw"
server_key: "8f4c91db5f4edf38b243a7cece2d51da"
server_key_iv: "b7761123df6c99c49ea923613fece180"
server_key_version: "encryptionkey@0"
I20260811 02:52:20.973111 20207 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786416707705494-12843-0/minicluster-data/ts-0/wal/instance:
uuid: "ab096c76c5544f828e9eb7f2bee1977e"
format_stamp: "Formatted at 2026-08-11 02:52:20 on dist-test-slave-9gcw"
server_key: "8f4c91db5f4edf38b243a7cece2d51da"
server_key_iv: "b7761123df6c99c49ea923613fece180"
server_key_version: "encryptionkey@0"
I20260811 02:52:20.976989 20207 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260811 02:52:20.979463 20226 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 02:52:20.980747 20207 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260811 02:52:20.980885 20207 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786416707705494-12843-0/minicluster-data/ts-0/data,/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786416707705494-12843-0/minicluster-data/ts-0/wal
uuid: "ab096c76c5544f828e9eb7f2bee1977e"
format_stamp: "Formatted at 2026-08-11 02:52:20 on dist-test-slave-9gcw"
server_key: "8f4c91db5f4edf38b243a7cece2d51da"
server_key_iv: "b7761123df6c99c49ea923613fece180"
server_key_version: "encryptionkey@0"
I20260811 02:52:20.980997 20207 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786416707705494-12843-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786416707705494-12843-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786416707705494-12843-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 02:52:20.990490 20207 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 02:52:20.993507 20207 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 02:52:20.993709 20207 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 02:52:20.994423 20207 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 02:52:20.995262 20207 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 02:52:20.995365 20207 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 02:52:20.995442 20207 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 02:52:20.995476 20207 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 02:52:21.004930 20207 rpc_server.cc:307] RPC server started. Bound to: 127.12.138.193:41461
I20260811 02:52:21.004952 20358 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.138.193:41461 every 8 connection(s)
I20260811 02:52:21.005991 20207 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786416707705494-12843-0/minicluster-data/ts-0/data/info.pb
I20260811 02:52:21.014621 12843 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDawEwS/build/debug/bin/kudu as pid 20207
I20260811 02:52:21.014783 12843 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786416707705494-12843-0/minicluster-data/ts-0/wal/instance
I20260811 02:52:21.015208 12843 external_mini_cluster.cc:1428] Setting key a566bbf17564f51298698de4e4077bf0
Aug 11 02:52:21 dist-test-slave-9gcw krb5kdc[20077](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416740, etypes {rep=17 tkt=17 ses=17}, oryx/127.12.138.193@KRBTEST.COM for oryx/127.12.138.254@KRBTEST.COM
I20260811 02:52:21.021445 20182 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:52:21.007865 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.138.193:59569 (local address 127.12.138.254:43457)
0811 02:52:21.008012 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0811 02:52:21.008016 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 02:52:21.008819 (+ 803us) server_negotiation.cc:407] Connection header received
0811 02:52:21.010316 (+ 1497us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:52:21.010320 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:52:21.010392 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:52:21.010499 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:52:21.011853 (+ 1354us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:52:21.012408 (+ 555us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:52:21.013806 (+ 1398us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:52:21.014056 (+ 250us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:52:21.014177 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:52:21.017170 (+ 2993us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:52:21.017175 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:52:21.017178 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:52:21.017205 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:52:21.018957 (+ 1752us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:52:21.019681 (+ 724us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:52:21.019683 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:52:21.019685 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:52:21.019764 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:52:21.020179 (+ 415us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:52:21.020181 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:52:21.020182 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:52:21.020363 (+ 181us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:52:21.020443 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0811 02:52:21.021085 (+ 642us) server_negotiation.cc:299] Negotiation successful
0811 02:52:21.021261 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260811 02:52:21.022282 20362 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:52:21.008181 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.138.254:43457 (local address 127.12.138.193:59569)
0811 02:52:21.008652 (+ 471us) negotiation.cc:107] Waiting for socket to connect
0811 02:52:21.008696 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0811 02:52:21.010082 (+ 1386us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:52:21.010682 (+ 600us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:52:21.010690 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:52:21.011150 (+ 460us) client_negotiation.cc:189] Negotiated authn=SASL
0811 02:52:21.011697 (+ 547us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:52:21.011712 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:52:21.012553 (+ 841us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:52:21.012556 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:52:21.013669 (+ 1113us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:52:21.013679 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:52:21.014219 (+ 540us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:52:21.014224 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:52:21.014485 (+ 261us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:52:21.015234 (+ 749us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 02:52:21.015257 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0811 02:52:21.016970 (+ 1713us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 02:52:21.019098 (+ 2128us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:52:21.019106 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:52:21.019125 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:52:21.019563 (+ 438us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:52:21.019867 (+ 304us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:52:21.019872 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:52:21.019876 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:52:21.020079 (+ 203us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:52:21.020478 (+ 399us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 02:52:21.020487 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 02:52:21.020829 (+ 342us) client_negotiation.cc:769] Sending connection context
0811 02:52:21.021035 (+ 206us) client_negotiation.cc:240] Negotiation successful
0811 02:52:21.021415 (+ 380us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":281,"thread_start_us":126,"threads_started":1}
I20260811 02:52:21.023618 20359 heartbeater.cc:347] Connected to a master server at 127.12.138.254:43457
I20260811 02:52:21.023919 20359 heartbeater.cc:464] Registering TS with master...
I20260811 02:52:21.024631 20359 heartbeater.cc:511] Master 127.12.138.254:43457 requested a full tablet report, sending...
I20260811 02:52:21.026525 20136 ts_manager.cc:194] Registered new tserver with Master: ab096c76c5544f828e9eb7f2bee1977e (127.12.138.193:41461)
WARNING: no policy specified for oryx/127.12.138.194@KRBTEST.COM; defaulting to no policy
I20260811 02:52:21.028138 20136 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.12.138.193@KRBTEST.COM'} at 127.12.138.193:59569
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.12.138.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.12.138.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786416707705494-12843-0/krb5kdc/oryx_127.12.138.194.keytab.
Entry for principal oryx/127.12.138.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786416707705494-12843-0/krb5kdc/oryx_127.12.138.194.keytab.
I20260811 02:52:21.048264 12843 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDawEwS/build/debug/bin/kudu
/tmp/dist-test-taskDawEwS/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786416707705494-12843-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786416707705494-12843-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786416707705494-12843-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786416707705494-12843-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.138.194:0
--local_ip_for_outbound_sockets=127.12.138.194
--webserver_interface=127.12.138.194
--webserver_port=0
--tserver_master_addrs=127.12.138.254:43457
--keytab_file=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786416707705494-12843-0/krb5kdc/oryx_127.12.138.194.keytab
--principal=oryx/127.12.138.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.138.212:41877
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786416707705494-12843-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786416707705494-12843-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786416707705494-12843-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 02:52:21.164072 20365 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 02:52:21.164335 20365 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 02:52:21.164378 20365 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 02:52:21.168470 20365 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 02:52:21.168531 20365 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 02:52:21.168589 20365 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.138.194
I20260811 02:52:21.173844 20365 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.138.212:41877
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786416707705494-12843-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786416707705494-12843-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786416707705494-12843-0/krb5kdc/oryx_127.12.138.194.keytab
--principal=oryx/127.12.138.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.138.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786416707705494-12843-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.12.138.194
--webserver_port=0
--tserver_master_addrs=127.12.138.254:43457
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20365
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.138.194
--log_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786416707705494-12843-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 02:09:49 UTC on ccycloud.quasar-fsnqqc.root.comops.site
build id 4579570
I20260811 02:52:21.175041 20365 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 02:52:21.176206 20365 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 02:52:21.185809 20375 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:52:21.185812 20371 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:52:21.185884 20372 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:52:21.186087 20365 server_base.cc:1034] running on GCE node
I20260811 02:52:21.186637 20365 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 02:52:21.187300 20365 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 02:52:21.188488 20365 hybrid_clock.cc:648] HybridClock initialized: now 1786416741188474 us; error 25 us; skew 500 ppm
Aug 11 02:52:21 dist-test-slave-9gcw krb5kdc[20077](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416741, etypes {rep=17 tkt=17 ses=17}, oryx/127.12.138.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 02:52:21.191461 20365 init.cc:377] Logged in from keytab as oryx/127.12.138.194@KRBTEST.COM (short username oryx)
I20260811 02:52:21.192425 20365 webserver.cc:460] Webserver started at http://127.12.138.194:39213/ using document root <none> and password file <none>
I20260811 02:52:21.193013 20365 fs_manager.cc:359] Metadata directory not provided
I20260811 02:52:21.193109 20365 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 02:52:21.193387 20365 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 02:52:21.195514 20365 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786416707705494-12843-0/minicluster-data/ts-1/data/instance:
uuid: "c51f9680930c4419a91d0d0f521006dd"
format_stamp: "Formatted at 2026-08-11 02:52:21 on dist-test-slave-9gcw"
server_key: "4aa288bff2395caa74179f6d82f5b9b3"
server_key_iv: "85708040c4669db6eba453f80cc6f18c"
server_key_version: "encryptionkey@0"
I20260811 02:52:21.196103 20365 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786416707705494-12843-0/minicluster-data/ts-1/wal/instance:
uuid: "c51f9680930c4419a91d0d0f521006dd"
format_stamp: "Formatted at 2026-08-11 02:52:21 on dist-test-slave-9gcw"
server_key: "4aa288bff2395caa74179f6d82f5b9b3"
server_key_iv: "85708040c4669db6eba453f80cc6f18c"
server_key_version: "encryptionkey@0"
I20260811 02:52:21.199869 20365 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.003s
I20260811 02:52:21.202441 20382 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 02:52:21.203619 20365 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260811 02:52:21.203814 20365 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786416707705494-12843-0/minicluster-data/ts-1/data,/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786416707705494-12843-0/minicluster-data/ts-1/wal
uuid: "c51f9680930c4419a91d0d0f521006dd"
format_stamp: "Formatted at 2026-08-11 02:52:21 on dist-test-slave-9gcw"
server_key: "4aa288bff2395caa74179f6d82f5b9b3"
server_key_iv: "85708040c4669db6eba453f80cc6f18c"
server_key_version: "encryptionkey@0"
I20260811 02:52:21.203931 20365 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786416707705494-12843-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786416707705494-12843-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786416707705494-12843-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 02:52:21.221632 20365 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 02:52:21.224467 20365 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 02:52:21.224689 20365 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 02:52:21.225294 20365 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 02:52:21.226135 20365 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 02:52:21.226207 20365 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 02:52:21.226279 20365 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 02:52:21.226325 20365 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 02:52:21.235404 20365 rpc_server.cc:307] RPC server started. Bound to: 127.12.138.194:40543
I20260811 02:52:21.235423 20511 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.138.194:40543 every 8 connection(s)
I20260811 02:52:21.236289 20365 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786416707705494-12843-0/minicluster-data/ts-1/data/info.pb
I20260811 02:52:21.246040 12843 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDawEwS/build/debug/bin/kudu as pid 20365
I20260811 02:52:21.246206 12843 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786416707705494-12843-0/minicluster-data/ts-1/wal/instance
Aug 11 02:52:21 dist-test-slave-9gcw krb5kdc[20077](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416741, etypes {rep=17 tkt=17 ses=17}, oryx/127.12.138.194@KRBTEST.COM for oryx/127.12.138.254@KRBTEST.COM
I20260811 02:52:21.246556 12843 external_mini_cluster.cc:1428] Setting key 6088a295d81376805e3db547a8df9399
I20260811 02:52:21.250870 20182 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:52:21.238100 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.138.194:46787 (local address 127.12.138.254:43457)
0811 02:52:21.238259 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0811 02:52:21.238264 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 02:52:21.239155 (+ 891us) server_negotiation.cc:407] Connection header received
0811 02:52:21.240491 (+ 1336us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:52:21.240494 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:52:21.240545 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:52:21.240624 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:52:21.241830 (+ 1206us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:52:21.242399 (+ 569us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:52:21.243619 (+ 1220us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:52:21.243919 (+ 300us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:52:21.244030 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:52:21.246987 (+ 2957us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:52:21.246993 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:52:21.246996 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:52:21.247033 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:52:21.248681 (+ 1648us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:52:21.249266 (+ 585us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:52:21.249270 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:52:21.249272 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:52:21.249375 (+ 103us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:52:21.249737 (+ 362us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:52:21.249740 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:52:21.249742 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:52:21.249953 (+ 211us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:52:21.250068 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0811 02:52:21.250579 (+ 511us) server_negotiation.cc:299] Negotiation successful
0811 02:52:21.250705 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260811 02:52:21.251677 20514 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:52:21.238552 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.138.254:43457 (local address 127.12.138.194:46787)
0811 02:52:21.238997 (+ 445us) negotiation.cc:107] Waiting for socket to connect
0811 02:52:21.239039 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0811 02:52:21.240268 (+ 1229us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:52:21.240797 (+ 529us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:52:21.240804 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:52:21.241192 (+ 388us) client_negotiation.cc:189] Negotiated authn=SASL
0811 02:52:21.241696 (+ 504us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:52:21.241709 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:52:21.242515 (+ 806us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:52:21.242517 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:52:21.243511 (+ 994us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:52:21.243521 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:52:21.244120 (+ 599us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:52:21.244124 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:52:21.244298 (+ 174us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:52:21.244990 (+ 692us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 02:52:21.245007 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 02:52:21.246797 (+ 1790us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 02:52:21.248818 (+ 2021us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:52:21.248824 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:52:21.248839 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:52:21.249103 (+ 264us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:52:21.249482 (+ 379us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:52:21.249485 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:52:21.249487 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:52:21.249629 (+ 142us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:52:21.250066 (+ 437us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 02:52:21.250074 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 02:52:21.250331 (+ 257us) client_negotiation.cc:769] Sending connection context
0811 02:52:21.250521 (+ 190us) client_negotiation.cc:240] Negotiation successful
0811 02:52:21.250862 (+ 341us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":258,"thread_start_us":125,"threads_started":1}
I20260811 02:52:21.253212 20512 heartbeater.cc:347] Connected to a master server at 127.12.138.254:43457
I20260811 02:52:21.253475 20512 heartbeater.cc:464] Registering TS with master...
I20260811 02:52:21.253983 20512 heartbeater.cc:511] Master 127.12.138.254:43457 requested a full tablet report, sending...
I20260811 02:52:21.255082 20136 ts_manager.cc:194] Registered new tserver with Master: c51f9680930c4419a91d0d0f521006dd (127.12.138.194:40543)
I20260811 02:52:21.255944 20136 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.12.138.194@KRBTEST.COM'} at 127.12.138.194:46787
WARNING: no policy specified for oryx/127.12.138.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.12.138.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.12.138.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786416707705494-12843-0/krb5kdc/oryx_127.12.138.195.keytab.
Entry for principal oryx/127.12.138.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786416707705494-12843-0/krb5kdc/oryx_127.12.138.195.keytab.
I20260811 02:52:21.278558 12843 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDawEwS/build/debug/bin/kudu
/tmp/dist-test-taskDawEwS/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786416707705494-12843-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786416707705494-12843-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786416707705494-12843-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786416707705494-12843-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.138.195:0
--local_ip_for_outbound_sockets=127.12.138.195
--webserver_interface=127.12.138.195
--webserver_port=0
--tserver_master_addrs=127.12.138.254:43457
--keytab_file=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786416707705494-12843-0/krb5kdc/oryx_127.12.138.195.keytab
--principal=oryx/127.12.138.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.138.212:41877
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786416707705494-12843-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786416707705494-12843-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786416707705494-12843-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 02:52:21.395601 20517 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 02:52:21.395915 20517 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 02:52:21.395991 20517 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 02:52:21.400050 20517 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 02:52:21.400135 20517 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 02:52:21.400225 20517 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.138.195
I20260811 02:52:21.405673 20517 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.138.212:41877
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786416707705494-12843-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786416707705494-12843-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786416707705494-12843-0/krb5kdc/oryx_127.12.138.195.keytab
--principal=oryx/127.12.138.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.138.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786416707705494-12843-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.12.138.195
--webserver_port=0
--tserver_master_addrs=127.12.138.254:43457
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20517
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.138.195
--log_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786416707705494-12843-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 02:09:49 UTC on ccycloud.quasar-fsnqqc.root.comops.site
build id 4579570
I20260811 02:52:21.406811 20517 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 02:52:21.407873 20517 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 02:52:21.417780 20525 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:52:21.417774 20524 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:52:21.417780 20527 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:52:21.418057 20517 server_base.cc:1034] running on GCE node
I20260811 02:52:21.418602 20517 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 02:52:21.419247 20517 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 02:52:21.420416 20517 hybrid_clock.cc:648] HybridClock initialized: now 1786416741420374 us; error 54 us; skew 500 ppm
Aug 11 02:52:21 dist-test-slave-9gcw krb5kdc[20077](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416741, etypes {rep=17 tkt=17 ses=17}, oryx/127.12.138.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 02:52:21.423246 20517 init.cc:377] Logged in from keytab as oryx/127.12.138.195@KRBTEST.COM (short username oryx)
I20260811 02:52:21.424191 20517 webserver.cc:460] Webserver started at http://127.12.138.195:42669/ using document root <none> and password file <none>
I20260811 02:52:21.424799 20517 fs_manager.cc:359] Metadata directory not provided
I20260811 02:52:21.424898 20517 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 02:52:21.425186 20517 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 02:52:21.427374 20517 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786416707705494-12843-0/minicluster-data/ts-2/data/instance:
uuid: "adde0ab9117a451c8cf06b46870c7c07"
format_stamp: "Formatted at 2026-08-11 02:52:21 on dist-test-slave-9gcw"
server_key: "99d255acabd2ed71a7cc41699f59754f"
server_key_iv: "04aaf406d1962249d1a260030a7b1add"
server_key_version: "encryptionkey@0"
I20260811 02:52:21.427974 20517 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786416707705494-12843-0/minicluster-data/ts-2/wal/instance:
uuid: "adde0ab9117a451c8cf06b46870c7c07"
format_stamp: "Formatted at 2026-08-11 02:52:21 on dist-test-slave-9gcw"
server_key: "99d255acabd2ed71a7cc41699f59754f"
server_key_iv: "04aaf406d1962249d1a260030a7b1add"
server_key_version: "encryptionkey@0"
I20260811 02:52:21.431890 20517 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.004s
I20260811 02:52:21.434475 20535 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 02:52:21.435667 20517 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260811 02:52:21.435823 20517 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786416707705494-12843-0/minicluster-data/ts-2/data,/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786416707705494-12843-0/minicluster-data/ts-2/wal
uuid: "adde0ab9117a451c8cf06b46870c7c07"
format_stamp: "Formatted at 2026-08-11 02:52:21 on dist-test-slave-9gcw"
server_key: "99d255acabd2ed71a7cc41699f59754f"
server_key_iv: "04aaf406d1962249d1a260030a7b1add"
server_key_version: "encryptionkey@0"
I20260811 02:52:21.435945 20517 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786416707705494-12843-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786416707705494-12843-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786416707705494-12843-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 02:52:21.448185 20517 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 02:52:21.451092 20517 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 02:52:21.451310 20517 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 02:52:21.451925 20517 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 02:52:21.452790 20517 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 02:52:21.452864 20517 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 02:52:21.452939 20517 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 02:52:21.452993 20517 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 02:52:21.462276 20517 rpc_server.cc:307] RPC server started. Bound to: 127.12.138.195:42681
I20260811 02:52:21.462307 20673 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.138.195:42681 every 8 connection(s)
I20260811 02:52:21.463238 20517 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786416707705494-12843-0/minicluster-data/ts-2/data/info.pb
I20260811 02:52:21.467182 12843 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDawEwS/build/debug/bin/kudu as pid 20517
I20260811 02:52:21.467293 12843 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786416707705494-12843-0/minicluster-data/ts-2/wal/instance
I20260811 02:52:21.467604 12843 external_mini_cluster.cc:1428] Setting key b3f87f8681f8c75b8de66b43b5735f65
Aug 11 02:52:21 dist-test-slave-9gcw krb5kdc[20077](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416741, etypes {rep=17 tkt=17 ses=17}, oryx/127.12.138.195@KRBTEST.COM for oryx/127.12.138.254@KRBTEST.COM
I20260811 02:52:21.477397 20182 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:52:21.465183 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.138.195:59913 (local address 127.12.138.254:43457)
0811 02:52:21.465349 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0811 02:52:21.465354 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 02:52:21.466336 (+ 982us) server_negotiation.cc:407] Connection header received
0811 02:52:21.467862 (+ 1526us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:52:21.467867 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:52:21.467928 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:52:21.468034 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:52:21.469494 (+ 1460us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:52:21.470204 (+ 710us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:52:21.471335 (+ 1131us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:52:21.471549 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:52:21.471626 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:52:21.474172 (+ 2546us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:52:21.474176 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:52:21.474178 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:52:21.474200 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:52:21.475493 (+ 1293us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:52:21.475988 (+ 495us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:52:21.475990 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:52:21.475992 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:52:21.476068 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:52:21.476371 (+ 303us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:52:21.476373 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:52:21.476374 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:52:21.476550 (+ 176us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:52:21.476625 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0811 02:52:21.477103 (+ 478us) server_negotiation.cc:299] Negotiation successful
0811 02:52:21.477265 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260811 02:52:21.478204 20678 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:52:21.465610 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.138.254:43457 (local address 127.12.138.195:59913)
0811 02:52:21.466155 (+ 545us) negotiation.cc:107] Waiting for socket to connect
0811 02:52:21.466201 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0811 02:52:21.467590 (+ 1389us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:52:21.468256 (+ 666us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:52:21.468263 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:52:21.468678 (+ 415us) client_negotiation.cc:189] Negotiated authn=SASL
0811 02:52:21.469357 (+ 679us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:52:21.469373 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:52:21.470323 (+ 950us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:52:21.470326 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:52:21.471228 (+ 902us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:52:21.471237 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:52:21.471675 (+ 438us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:52:21.471678 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:52:21.471832 (+ 154us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:52:21.472474 (+ 642us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 02:52:21.472491 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 02:52:21.474018 (+ 1527us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 02:52:21.475601 (+ 1583us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:52:21.475606 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:52:21.475618 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:52:21.475885 (+ 267us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:52:21.476147 (+ 262us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:52:21.476150 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:52:21.476151 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:52:21.476282 (+ 131us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:52:21.476635 (+ 353us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 02:52:21.476640 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 02:52:21.476889 (+ 249us) client_negotiation.cc:769] Sending connection context
0811 02:52:21.477041 (+ 152us) client_negotiation.cc:240] Negotiation successful
0811 02:52:21.477364 (+ 323us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":290,"thread_start_us":142,"threads_started":1}
I20260811 02:52:21.479393 20674 heartbeater.cc:347] Connected to a master server at 127.12.138.254:43457
I20260811 02:52:21.479642 20674 heartbeater.cc:464] Registering TS with master...
I20260811 02:52:21.480150 20674 heartbeater.cc:511] Master 127.12.138.254:43457 requested a full tablet report, sending...
I20260811 02:52:21.481446 20136 ts_manager.cc:194] Registered new tserver with Master: adde0ab9117a451c8cf06b46870c7c07 (127.12.138.195:42681)
I20260811 02:52:21.481952 12843 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 02:52:21.482260 20136 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.12.138.195@KRBTEST.COM'} at 127.12.138.195:59913
WARNING: no policy specified for oryx/127.12.138.253@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.12.138.253@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.12.138.253 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786416707705494-12843-0/krb5kdc/oryx_127.12.138.253.keytab.
Entry for principal oryx/127.12.138.253 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786416707705494-12843-0/krb5kdc/oryx_127.12.138.253.keytab.
I20260811 02:52:21.512511 12843 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDawEwS/build/debug/bin/kudu
/tmp/dist-test-taskDawEwS/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786416707705494-12843-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786416707705494-12843-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786416707705494-12843-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786416707705494-12843-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.138.253:42467
--webserver_interface=127.12.138.253
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786416707705494-12843-0/krb5kdc/oryx_127.12.138.253.keytab
--principal=oryx/127.12.138.253
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.138.212:41877
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.12.138.254:43457,127.12.138.253:42467
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786416707705494-12843-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786416707705494-12843-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786416707705494-12843-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 02:52:21.627679 20683 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 02:52:21.628000 20683 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 02:52:21.628042 20683 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 02:52:21.632174 20683 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 02:52:21.632252 20683 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 02:52:21.632272 20683 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 02:52:21.632289 20683 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 02:52:21.632305 20683 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 02:52:21.637871 20683 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.138.212:41877
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786416707705494-12843-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786416707705494-12843-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.12.138.254:43457,127.12.138.253:42467
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786416707705494-12843-0/krb5kdc/oryx_127.12.138.253.keytab
--principal=oryx/127.12.138.253
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.138.253:42467
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786416707705494-12843-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.12.138.253
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20683
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786416707705494-12843-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 02:09:49 UTC on ccycloud.quasar-fsnqqc.root.comops.site
build id 4579570
W20260811 02:52:21.638095 20683 master_options.cc:55] Only 2 masters are specified by master_addresses_flag ('127.12.138.254:43457,127.12.138.253:42467'), but minimum of 3 are required to tolerate failures of any one master. It is recommended to use at least 3 masters.
I20260811 02:52:21.639012 20683 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 02:52:21.640067 20683 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 02:52:21.649070 20692 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:52:21.649101 20689 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:52:21.649060 20688 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:52:21.649420 20683 server_base.cc:1034] running on GCE node
I20260811 02:52:21.649938 20683 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 02:52:21.651072 20683 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 02:52:21.652261 20683 hybrid_clock.cc:648] HybridClock initialized: now 1786416741652245 us; error 27 us; skew 500 ppm
Aug 11 02:52:21 dist-test-slave-9gcw krb5kdc[20077](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416741, etypes {rep=17 tkt=17 ses=17}, oryx/127.12.138.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 02:52:21.655167 20683 init.cc:377] Logged in from keytab as oryx/127.12.138.253@KRBTEST.COM (short username oryx)
I20260811 02:52:21.656177 20683 webserver.cc:460] Webserver started at http://127.12.138.253:37897/ using document root <none> and password file <none>
I20260811 02:52:21.656744 20683 fs_manager.cc:359] Metadata directory not provided
I20260811 02:52:21.656849 20683 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 02:52:21.657094 20683 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 02:52:21.659168 20683 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786416707705494-12843-0/minicluster-data/master-1/data/instance:
uuid: "269d7f529b994b32bc6d6d1b512371d4"
format_stamp: "Formatted at 2026-08-11 02:52:21 on dist-test-slave-9gcw"
server_key: "5c2eb3a03a3916185234c7b90553fb11"
server_key_iv: "c542d4c823346f3e99a039bd3f75d72a"
server_key_version: "encryptionkey@0"
I20260811 02:52:21.659747 20683 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786416707705494-12843-0/minicluster-data/master-1/wal/instance:
uuid: "269d7f529b994b32bc6d6d1b512371d4"
format_stamp: "Formatted at 2026-08-11 02:52:21 on dist-test-slave-9gcw"
server_key: "5c2eb3a03a3916185234c7b90553fb11"
server_key_iv: "c542d4c823346f3e99a039bd3f75d72a"
server_key_version: "encryptionkey@0"
I20260811 02:52:21.663550 20683 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260811 02:52:21.666185 20701 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 02:52:21.667438 20683 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260811 02:52:21.667611 20683 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786416707705494-12843-0/minicluster-data/master-1/data,/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786416707705494-12843-0/minicluster-data/master-1/wal
uuid: "269d7f529b994b32bc6d6d1b512371d4"
format_stamp: "Formatted at 2026-08-11 02:52:21 on dist-test-slave-9gcw"
server_key: "5c2eb3a03a3916185234c7b90553fb11"
server_key_iv: "c542d4c823346f3e99a039bd3f75d72a"
server_key_version: "encryptionkey@0"
I20260811 02:52:21.667738 20683 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786416707705494-12843-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786416707705494-12843-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786416707705494-12843-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 02:52:21.679633 20683 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 02:52:21.682467 20683 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 02:52:21.682672 20683 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 02:52:21.690208 20683 rpc_server.cc:307] RPC server started. Bound to: 127.12.138.253:42467
I20260811 02:52:21.690594 20759 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.138.253:42467 every 8 connection(s)
I20260811 02:52:21.691287 20683 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786416707705494-12843-0/minicluster-data/master-1/data/info.pb
I20260811 02:52:21.694295 20760 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 02:52:21.698181 20760 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.12.138.254" port: 43457 } has no permanent_uuid. Determining permanent_uuid...
I20260811 02:52:21.699749 12843 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDawEwS/build/debug/bin/kudu as pid 20683
Aug 11 02:52:21 dist-test-slave-9gcw krb5kdc[20077](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416741, etypes {rep=17 tkt=17 ses=17}, oryx/127.12.138.253@KRBTEST.COM for oryx/127.12.138.254@KRBTEST.COM
Aug 11 02:52:21 dist-test-slave-9gcw krb5kdc[20077](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416740, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.12.138.253@KRBTEST.COM
I20260811 02:52:21.712714 20182 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:52:21.700206 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54546 (local address 127.12.138.254:43457)
0811 02:52:21.700373 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0811 02:52:21.700379 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 02:52:21.701752 (+ 1373us) server_negotiation.cc:407] Connection header received
0811 02:52:21.702452 (+ 700us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:52:21.702455 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:52:21.702503 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:52:21.702581 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:52:21.703583 (+ 1002us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:52:21.704098 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:52:21.705340 (+ 1242us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:52:21.705557 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:52:21.705636 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:52:21.708443 (+ 2807us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:52:21.708447 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:52:21.708449 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:52:21.708475 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:52:21.710451 (+ 1976us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:52:21.711044 (+ 593us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:52:21.711048 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:52:21.711049 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:52:21.711152 (+ 103us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:52:21.711497 (+ 345us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:52:21.711502 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:52:21.711504 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:52:21.711751 (+ 247us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:52:21.711868 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0811 02:52:21.712397 (+ 529us) server_negotiation.cc:299] Negotiation successful
0811 02:52:21.712530 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260811 02:52:21.713768 20763 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:52:21.700656 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.138.254:43457 (local address 127.0.0.1:54546)
0811 02:52:21.701606 (+ 950us) negotiation.cc:107] Waiting for socket to connect
0811 02:52:21.701653 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0811 02:52:21.702227 (+ 574us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:52:21.702656 (+ 429us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:52:21.702661 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:52:21.703122 (+ 461us) client_negotiation.cc:189] Negotiated authn=SASL
0811 02:52:21.703432 (+ 310us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:52:21.703445 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:52:21.704229 (+ 784us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:52:21.704233 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:52:21.705191 (+ 958us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:52:21.705203 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:52:21.705697 (+ 494us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:52:21.705701 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:52:21.705875 (+ 174us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:52:21.706648 (+ 773us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 02:52:21.706665 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 02:52:21.708286 (+ 1621us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 02:52:21.710578 (+ 2292us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:52:21.710585 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:52:21.710594 (+ 9us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:52:21.710938 (+ 344us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:52:21.711243 (+ 305us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:52:21.711251 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:52:21.711253 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:52:21.711393 (+ 140us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:52:21.711878 (+ 485us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 02:52:21.711887 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 02:52:21.712162 (+ 275us) client_negotiation.cc:769] Sending connection context
0811 02:52:21.712344 (+ 182us) client_negotiation.cc:240] Negotiation successful
0811 02:52:21.712686 (+ 342us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":749,"mutex_wait_us":25,"thread_start_us":152,"threads_started":1}
I20260811 02:52:21.713768 20764 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:52:21.700784 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60346 (local address 127.12.138.253:42467)
0811 02:52:21.701260 (+ 476us) server_negotiation.cc:206] Beginning negotiation
0811 02:52:21.701279 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0811 02:52:21.701344 (+ 65us) server_negotiation.cc:407] Connection header received
0811 02:52:21.702227 (+ 883us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:52:21.702252 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:52:21.702493 (+ 241us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:52:21.702693 (+ 200us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:52:21.703436 (+ 743us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:52:21.704439 (+ 1003us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:52:21.705339 (+ 900us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:52:21.705761 (+ 422us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:52:21.705881 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:52:21.709036 (+ 3155us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:52:21.709045 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:52:21.709058 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:52:21.709088 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:52:21.711050 (+ 1962us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:52:21.711481 (+ 431us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:52:21.711486 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:52:21.711488 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:52:21.711573 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:52:21.711863 (+ 290us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:52:21.711868 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:52:21.711870 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:52:21.712150 (+ 280us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:52:21.712369 (+ 219us) server_negotiation.cc:1035] Waiting for connection context
0811 02:52:21.712617 (+ 248us) server_negotiation.cc:299] Negotiation successful
0811 02:52:21.713020 (+ 403us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":330,"spinlock_wait_cycles":54656,"thread_start_us":152,"threads_started":1}
I20260811 02:52:21.715380 20760 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.12.138.253" port: 42467 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 02:52:21 dist-test-slave-9gcw krb5kdc[20077](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416741, etypes {rep=17 tkt=17 ses=17}, oryx/127.12.138.253@KRBTEST.COM for oryx/127.12.138.253@KRBTEST.COM
I20260811 02:52:21.724009 20763 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:52:21.715932 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.138.253:42467 (local address 127.0.0.1:60358)
0811 02:52:21.716057 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0811 02:52:21.716073 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 02:52:21.716146 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:52:21.716407 (+ 261us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:52:21.716409 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:52:21.716536 (+ 127us) client_negotiation.cc:189] Negotiated authn=SASL
0811 02:52:21.716624 (+ 88us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:52:21.716629 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:52:21.717362 (+ 733us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:52:21.717364 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:52:21.717863 (+ 499us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:52:21.717870 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:52:21.718980 (+ 1110us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:52:21.718984 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:52:21.719083 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:52:21.719552 (+ 469us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 02:52:21.719565 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0811 02:52:21.720906 (+ 1341us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 02:52:21.722516 (+ 1610us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:52:21.722522 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:52:21.722525 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:52:21.722717 (+ 192us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:52:21.723006 (+ 289us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:52:21.723009 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:52:21.723011 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:52:21.723090 (+ 79us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:52:21.723449 (+ 359us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 02:52:21.723452 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 02:52:21.723540 (+ 88us) client_negotiation.cc:769] Sending connection context
0811 02:52:21.723645 (+ 105us) client_negotiation.cc:240] Negotiation successful
0811 02:52:21.723782 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":51}
I20260811 02:52:21.724009 20764 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:52:21.716113 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60358 (local address 127.12.138.253:42467)
0811 02:52:21.716235 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0811 02:52:21.716238 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 02:52:21.716249 (+ 11us) server_negotiation.cc:407] Connection header received
0811 02:52:21.716277 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:52:21.716280 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:52:21.716313 (+ 33us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:52:21.716378 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:52:21.716715 (+ 337us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:52:21.717270 (+ 555us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:52:21.717958 (+ 688us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:52:21.718862 (+ 904us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:52:21.718949 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:52:21.721033 (+ 2084us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:52:21.721037 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:52:21.721041 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:52:21.721066 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:52:21.722403 (+ 1337us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:52:21.722838 (+ 435us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:52:21.722841 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:52:21.722842 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:52:21.722909 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:52:21.723176 (+ 267us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:52:21.723181 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:52:21.723183 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:52:21.723346 (+ 163us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:52:21.723450 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0811 02:52:21.723690 (+ 240us) server_negotiation.cc:299] Negotiation successful
0811 02:52:21.723844 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":60,"server-negotiator.queue_time_us":50}
I20260811 02:52:21.727353 20760 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 269d7f529b994b32bc6d6d1b512371d4: Bootstrap starting.
I20260811 02:52:21.730113 20760 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 269d7f529b994b32bc6d6d1b512371d4: Neither blocks nor log segments found. Creating new log.
I20260811 02:52:21.730963 20760 log.cc:824] T 00000000000000000000000000000000 P 269d7f529b994b32bc6d6d1b512371d4: Log is configured to *not* fsync() on all Append() calls
I20260811 02:52:21.733237 20760 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 269d7f529b994b32bc6d6d1b512371d4: No bootstrap required, opened a new log
I20260811 02:52:21.736179 20760 raft_consensus.cc:348] T 00000000000000000000000000000000 P 269d7f529b994b32bc6d6d1b512371d4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0317e693c61849e9bf42d55fbf6ef321" member_type: VOTER last_known_addr { host: "127.12.138.254" port: 43457 } } peers { permanent_uuid: "269d7f529b994b32bc6d6d1b512371d4" member_type: VOTER last_known_addr { host: "127.12.138.253" port: 42467 } }
I20260811 02:52:21.736389 20760 raft_consensus.cc:374] T 00000000000000000000000000000000 P 269d7f529b994b32bc6d6d1b512371d4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 02:52:21.736429 20760 raft_consensus.cc:729] T 00000000000000000000000000000000 P 269d7f529b994b32bc6d6d1b512371d4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 269d7f529b994b32bc6d6d1b512371d4, State: Initialized, Role: FOLLOWER
I20260811 02:52:21.736800 20760 consensus_queue.cc:260] T 00000000000000000000000000000000 P 269d7f529b994b32bc6d6d1b512371d4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0317e693c61849e9bf42d55fbf6ef321" member_type: VOTER last_known_addr { host: "127.12.138.254" port: 43457 } } peers { permanent_uuid: "269d7f529b994b32bc6d6d1b512371d4" member_type: VOTER last_known_addr { host: "127.12.138.253" port: 42467 } }
I20260811 02:52:21.737299 20767 sys_catalog.cc:455] T 00000000000000000000000000000000 P 269d7f529b994b32bc6d6d1b512371d4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0317e693c61849e9bf42d55fbf6ef321" member_type: VOTER last_known_addr { host: "127.12.138.254" port: 43457 } } peers { permanent_uuid: "269d7f529b994b32bc6d6d1b512371d4" member_type: VOTER last_known_addr { host: "127.12.138.253" port: 42467 } } }
I20260811 02:52:21.737416 20767 sys_catalog.cc:458] T 00000000000000000000000000000000 P 269d7f529b994b32bc6d6d1b512371d4 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 02:52:21.738061 20760 sys_catalog.cc:564] T 00000000000000000000000000000000 P 269d7f529b994b32bc6d6d1b512371d4 [sys.catalog]: configured and running, proceeding with master startup.
W20260811 02:52:21.743072 20779 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 269d7f529b994b32bc6d6d1b512371d4: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 02:52:21.743175 20779 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 02:52:21.745724 20683 master_runner.cc:428] Detected that this master 269d7f529b994b32bc6d6d1b512371d4 is joining an existing cluster
I20260811 02:52:21.745779 20683 master_runner.cc:432] Initiating AddMaster RPC to add 127.12.138.253:42467
I20260811 02:52:21.756834 20182 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:52:21.747937 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54548 (local address 127.12.138.254:43457)
0811 02:52:21.748066 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0811 02:52:21.748071 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 02:52:21.748340 (+ 269us) server_negotiation.cc:407] Connection header received
0811 02:52:21.748544 (+ 204us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:52:21.748547 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:52:21.748600 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:52:21.748685 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:52:21.749398 (+ 713us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:52:21.750154 (+ 756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:52:21.750995 (+ 841us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:52:21.751302 (+ 307us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:52:21.751404 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:52:21.753289 (+ 1885us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:52:21.753297 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:52:21.753303 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:52:21.753336 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:52:21.755017 (+ 1681us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:52:21.755571 (+ 554us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:52:21.755576 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:52:21.755580 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:52:21.755683 (+ 103us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:52:21.755966 (+ 283us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:52:21.755969 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:52:21.755971 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:52:21.756159 (+ 188us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:52:21.756281 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0811 02:52:21.756560 (+ 279us) server_negotiation.cc:299] Negotiation successful
0811 02:52:21.756675 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260811 02:52:21.756850 20786 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:52:21.747980 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.138.253:42467 (local address 127.0.0.1:60366)
0811 02:52:21.748250 (+ 270us) negotiation.cc:107] Waiting for socket to connect
0811 02:52:21.748266 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 02:52:21.748433 (+ 167us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:52:21.748750 (+ 317us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:52:21.748751 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:52:21.748964 (+ 213us) client_negotiation.cc:189] Negotiated authn=SASL
0811 02:52:21.749088 (+ 124us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:52:21.749094 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:52:21.750150 (+ 1056us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:52:21.750153 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:52:21.751080 (+ 927us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:52:21.751090 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:52:21.751632 (+ 542us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:52:21.751635 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:52:21.751731 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:52:21.752378 (+ 647us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 02:52:21.752390 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0811 02:52:21.752914 (+ 524us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 02:52:21.755039 (+ 2125us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:52:21.755043 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:52:21.755045 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:52:21.755404 (+ 359us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:52:21.755725 (+ 321us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:52:21.755728 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:52:21.755732 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:52:21.755829 (+ 97us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:52:21.756258 (+ 429us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 02:52:21.756262 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 02:52:21.756361 (+ 99us) client_negotiation.cc:769] Sending connection context
0811 02:52:21.756477 (+ 116us) client_negotiation.cc:240] Negotiation successful
0811 02:52:21.756634 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":227,"mutex_wait_us":3,"thread_start_us":93,"threads_started":1}
I20260811 02:52:21.756860 20764 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:52:21.748277 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60366 (local address 127.12.138.253:42467)
0811 02:52:21.748382 (+ 105us) server_negotiation.cc:206] Beginning negotiation
0811 02:52:21.748386 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 02:52:21.748397 (+ 11us) server_negotiation.cc:407] Connection header received
0811 02:52:21.748527 (+ 130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:52:21.748530 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:52:21.748570 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:52:21.748615 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:52:21.749233 (+ 618us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:52:21.750018 (+ 785us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:52:21.751199 (+ 1181us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:52:21.751511 (+ 312us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:52:21.751631 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:52:21.753045 (+ 1414us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:52:21.753049 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:52:21.753052 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:52:21.753074 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:52:21.754914 (+ 1840us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:52:21.755526 (+ 612us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:52:21.755532 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:52:21.755536 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:52:21.755619 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:52:21.755930 (+ 311us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:52:21.755933 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:52:21.755935 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:52:21.756146 (+ 211us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:52:21.756245 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0811 02:52:21.756537 (+ 292us) server_negotiation.cc:299] Negotiation successful
0811 02:52:21.756698 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":48,"server-negotiator.queue_time_us":48}
I20260811 02:52:21.757388 20787 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:52:21.747882 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.138.254:43457 (local address 127.0.0.1:54548)
0811 02:52:21.748250 (+ 368us) negotiation.cc:107] Waiting for socket to connect
0811 02:52:21.748266 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 02:52:21.748417 (+ 151us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:52:21.748813 (+ 396us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:52:21.748816 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:52:21.749048 (+ 232us) client_negotiation.cc:189] Negotiated authn=SASL
0811 02:52:21.749243 (+ 195us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:52:21.749251 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:52:21.750272 (+ 1021us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:52:21.750282 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:52:21.750867 (+ 585us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:52:21.750875 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:52:21.751438 (+ 563us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:52:21.751442 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:52:21.751554 (+ 112us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:52:21.752574 (+ 1020us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 02:52:21.752589 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0811 02:52:21.753136 (+ 547us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 02:52:21.755106 (+ 1970us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:52:21.755109 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:52:21.755111 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:52:21.755418 (+ 307us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:52:21.755762 (+ 344us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:52:21.755767 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:52:21.755772 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:52:21.755846 (+ 74us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:52:21.756321 (+ 475us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 02:52:21.756324 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 02:52:21.756386 (+ 62us) client_negotiation.cc:769] Sending connection context
0811 02:52:21.757050 (+ 664us) client_negotiation.cc:240] Negotiation successful
0811 02:52:21.757208 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":285,"mutex_wait_us":24,"thread_start_us":117,"threads_started":1}
Aug 11 02:52:21 dist-test-slave-9gcw krb5kdc[20077](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416740, etypes {rep=17 tkt=17 ses=17}, oryx/127.12.138.254@KRBTEST.COM for oryx/127.12.138.253@KRBTEST.COM
I20260811 02:52:21.771787 20764 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:52:21.761783 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60368 (local address 127.12.138.253:42467)
0811 02:52:21.761902 (+ 119us) server_negotiation.cc:206] Beginning negotiation
0811 02:52:21.761907 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 02:52:21.762213 (+ 306us) server_negotiation.cc:407] Connection header received
0811 02:52:21.762441 (+ 228us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:52:21.762444 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:52:21.762500 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:52:21.762588 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:52:21.763240 (+ 652us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:52:21.763730 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:52:21.764637 (+ 907us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:52:21.764874 (+ 237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:52:21.764989 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:52:21.767870 (+ 2881us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:52:21.767874 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:52:21.767877 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:52:21.767902 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:52:21.769650 (+ 1748us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:52:21.770296 (+ 646us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:52:21.770300 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:52:21.770302 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:52:21.770404 (+ 102us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:52:21.770700 (+ 296us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:52:21.770705 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:52:21.770706 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:52:21.770975 (+ 269us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:52:21.771096 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0811 02:52:21.771482 (+ 386us) server_negotiation.cc:299] Negotiation successful
0811 02:52:21.771609 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260811 02:52:21.771802 20788 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:52:21.761743 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.138.253:42467 (local address 127.0.0.1:60368)
0811 02:52:21.762093 (+ 350us) negotiation.cc:107] Waiting for socket to connect
0811 02:52:21.762136 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0811 02:52:21.762306 (+ 170us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:52:21.762625 (+ 319us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:52:21.762631 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:52:21.763007 (+ 376us) client_negotiation.cc:189] Negotiated authn=SASL
0811 02:52:21.763116 (+ 109us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:52:21.763122 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:52:21.763847 (+ 725us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:52:21.763852 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:52:21.764508 (+ 656us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:52:21.764518 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:52:21.764993 (+ 475us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:52:21.764997 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:52:21.765121 (+ 124us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:52:21.765717 (+ 596us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 02:52:21.765729 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0811 02:52:21.767721 (+ 1992us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 02:52:21.769794 (+ 2073us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:52:21.769809 (+ 15us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:52:21.769820 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:52:21.770190 (+ 370us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:52:21.770505 (+ 315us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:52:21.770509 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:52:21.770511 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:52:21.770617 (+ 106us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:52:21.771097 (+ 480us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 02:52:21.771107 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 02:52:21.771257 (+ 150us) client_negotiation.cc:769] Sending connection context
0811 02:52:21.771428 (+ 171us) client_negotiation.cc:240] Negotiation successful
0811 02:52:21.771647 (+ 219us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":193,"thread_start_us":100,"threads_started":1}
I20260811 02:52:21.773738 20136 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.12.138.253:42467: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "269d7f529b994b32bc6d6d1b512371d4"
member_type: NON_VOTER
last_known_addr {
host: "127.12.138.253"
port: 42467
}
attrs {
promote: true
}
}
dest_uuid: "0317e693c61849e9bf42d55fbf6ef321"
cas_config_opid_index: -1
I20260811 02:52:21.774477 20136 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0317e693c61849e9bf42d55fbf6ef321 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "0317e693c61849e9bf42d55fbf6ef321" member_type: VOTER last_known_addr { host: "127.12.138.254" port: 43457 } } peers { permanent_uuid: "269d7f529b994b32bc6d6d1b512371d4" member_type: NON_VOTER last_known_addr { host: "127.12.138.253" port: 42467 } attrs { promote: true } }
I20260811 02:52:21.776185 20791 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 0317e693c61849e9bf42d55fbf6ef321 [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER 269d7f529b994b32bc6d6d1b512371d4 (127.12.138.253) added. New config: { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "0317e693c61849e9bf42d55fbf6ef321" member_type: VOTER last_known_addr { host: "127.12.138.254" port: 43457 } } peers { permanent_uuid: "269d7f529b994b32bc6d6d1b512371d4" member_type: NON_VOTER last_known_addr { host: "127.12.138.253" port: 42467 } attrs { promote: true } } }
I20260811 02:52:21.777282 20733 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 269d7f529b994b32bc6d6d1b512371d4 [term 0 FOLLOWER]: Advancing to term 1
WARNING: no policy specified for oryx/127.12.138.252@KRBTEST.COM; defaulting to no policy
I20260811 02:52:21.778702 20791 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.12.138.253:42467
I20260811 02:52:21.778798 20733 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 269d7f529b994b32bc6d6d1b512371d4 [term 1 FOLLOWER]: Refusing update from remote peer 0317e693c61849e9bf42d55fbf6ef321: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20260811 02:52:21.779330 20683 master.cc:537] Master@127.12.138.253:42467 shutting down...
I20260811 02:52:21.779366 20767 raft_consensus.cc:482] T 00000000000000000000000000000000 P 269d7f529b994b32bc6d6d1b512371d4 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 02:52:21.779542 20767 raft_consensus.cc:504] T 00000000000000000000000000000000 P 269d7f529b994b32bc6d6d1b512371d4 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0317e693c61849e9bf42d55fbf6ef321" member_type: VOTER last_known_addr { host: "127.12.138.254" port: 43457 } } peers { permanent_uuid: "269d7f529b994b32bc6d6d1b512371d4" member_type: VOTER last_known_addr { host: "127.12.138.253" port: 42467 } }
I20260811 02:52:21.779805 20792 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0317e693c61849e9bf42d55fbf6ef321 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "0317e693c61849e9bf42d55fbf6ef321" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "0317e693c61849e9bf42d55fbf6ef321" member_type: VOTER last_known_addr { host: "127.12.138.254" port: 43457 } } peers { permanent_uuid: "269d7f529b994b32bc6d6d1b512371d4" member_type: NON_VOTER last_known_addr { host: "127.12.138.253" port: 42467 } attrs { promote: true } } }
I20260811 02:52:21.779927 20792 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0317e693c61849e9bf42d55fbf6ef321 [sys.catalog]: This master's current role is: LEADER
I20260811 02:52:21.779829 20791 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 0317e693c61849e9bf42d55fbf6ef321 [LEADER]: Connected to new peer: Peer: permanent_uuid: "269d7f529b994b32bc6d6d1b512371d4" member_type: NON_VOTER last_known_addr { host: "127.12.138.253" port: 42467 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 02:52:21.780421 20767 leader_election.cc:288] T 00000000000000000000000000000000 P 269d7f529b994b32bc6d6d1b512371d4 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0317e693c61849e9bf42d55fbf6ef321 (127.12.138.254:43457)
I20260811 02:52:21.780856 20143 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "269d7f529b994b32bc6d6d1b512371d4" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0317e693c61849e9bf42d55fbf6ef321" is_pre_election: true
I20260811 02:52:21.781540 20702 leader_election.cc:302] T 00000000000000000000000000000000 P 269d7f529b994b32bc6d6d1b512371d4 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 2 responses out of 2 voters: 1 yes votes; 1 no votes. yes voters: 269d7f529b994b32bc6d6d1b512371d4; no voters: 0317e693c61849e9bf42d55fbf6ef321
I20260811 02:52:21.781819 20767 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 269d7f529b994b32bc6d6d1b512371d4 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260811 02:52:21.782692 20115 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master
W20260811 02:52:21.783751 20115 consensus_peers.cc:487] T 00000000000000000000000000000000 P 0317e693c61849e9bf42d55fbf6ef321 -> Peer 269d7f529b994b32bc6d6d1b512371d4 (127.12.138.253:42467): Couldn't send request to peer 269d7f529b994b32bc6d6d1b512371d4. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.12.138.252@KRBTEST.COM" created.
I20260811 02:52:21.785792 20683 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 269d7f529b994b32bc6d6d1b512371d4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 02:52:21.786063 20683 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 269d7f529b994b32bc6d6d1b512371d4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 02:52:21.786158 20683 tablet_replica.cc:330] T 00000000000000000000000000000000 P 269d7f529b994b32bc6d6d1b512371d4: stopping tablet replica
I20260811 02:52:21.791061 20683 master.cc:552] Master@127.12.138.253:42467 shutdown complete.
I20260811 02:52:21.791121 20683 master_runner.cc:305] Clearing existing system tablet
I20260811 02:52:21.791386 20683 fs_manager.cc:359] Metadata directory not provided
I20260811 02:52:21.791446 20683 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 02:52:21.792784 20683 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.002s sys 0.000s
I20260811 02:52:21.793771 20798 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 02:52:21.794062 20683 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786416707705494-12843-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 02:52:21.794188 20683 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260811 02:52:21.794229 20683 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786416707705494-12843-0/minicluster-data/master-1/data,/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786416707705494-12843-0/minicluster-data/master-1/wal
uuid: "269d7f529b994b32bc6d6d1b512371d4"
format_stamp: "Formatted at 2026-08-11 02:52:21 on dist-test-slave-9gcw"
server_key: "5c2eb3a03a3916185234c7b90553fb11"
server_key_iv: "c542d4c823346f3e99a039bd3f75d72a"
server_key_version: "encryptionkey@0"
I20260811 02:52:21.796434 20683 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P 269d7f529b994b32bc6d6d1b512371d4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 02:52:21.799846 20683 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P 269d7f529b994b32bc6d6d1b512371d4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260811 02:52:21.799947 20683 log.cc:1196] T 00000000000000000000000000000000 P 269d7f529b994b32bc6d6d1b512371d4: Deleting WAL directory at /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786416707705494-12843-0/minicluster-data/master-1/wal/wals/00000000000000000000000000000000
I20260811 02:52:21.800204 20683 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P 269d7f529b994b32bc6d6d1b512371d4: Deleting consensus metadata
I20260811 02:52:21.800391 20683 master_runner.cc:315] Copying system tablet from 127.12.138.254:43457
I20260811 02:52:21.801741 20683 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P 269d7f529b994b32bc6d6d1b512371d4: tablet copy: Beginning tablet copy session from remote peer at address 127.12.138.254:43457
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.12.138.252 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786416707705494-12843-0/krb5kdc/oryx_127.12.138.252.keytab.
Entry for principal oryx/127.12.138.252 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786416707705494-12843-0/krb5kdc/oryx_127.12.138.252.keytab.
I20260811 02:52:21.805510 12843 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDawEwS/build/debug/bin/kudu
/tmp/dist-test-taskDawEwS/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786416707705494-12843-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786416707705494-12843-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786416707705494-12843-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786416707705494-12843-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.138.252:42917
--webserver_interface=127.12.138.252
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786416707705494-12843-0/krb5kdc/oryx_127.12.138.252.keytab
--principal=oryx/127.12.138.252
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.138.212:41877
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.12.138.254:43457,127.12.138.253:42467,127.12.138.252:42917
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786416707705494-12843-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786416707705494-12843-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786416707705494-12843-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 02:52:21.811308 20804 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:52:21.802329 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.138.254:43457 (local address 127.0.0.1:54560)
0811 02:52:21.802628 (+ 299us) negotiation.cc:107] Waiting for socket to connect
0811 02:52:21.802644 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 02:52:21.802770 (+ 126us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:52:21.803058 (+ 288us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:52:21.803061 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:52:21.803282 (+ 221us) client_negotiation.cc:189] Negotiated authn=SASL
0811 02:52:21.803410 (+ 128us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:52:21.803417 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:52:21.804414 (+ 997us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:52:21.804417 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:52:21.805229 (+ 812us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:52:21.805243 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:52:21.806068 (+ 825us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:52:21.806072 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:52:21.806172 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:52:21.806818 (+ 646us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 02:52:21.806834 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0811 02:52:21.807273 (+ 439us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 02:52:21.809459 (+ 2186us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:52:21.809465 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:52:21.809470 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:52:21.809785 (+ 315us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:52:21.810095 (+ 310us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:52:21.810101 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:52:21.810106 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:52:21.810233 (+ 127us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:52:21.810717 (+ 484us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 02:52:21.810720 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 02:52:21.810821 (+ 101us) client_negotiation.cc:769] Sending connection context
0811 02:52:21.810928 (+ 107us) client_negotiation.cc:240] Negotiation successful
0811 02:52:21.811087 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":218,"thread_start_us":101,"threads_started":1}
I20260811 02:52:21.811417 20182 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:52:21.802466 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54560 (local address 127.12.138.254:43457)
0811 02:52:21.802613 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0811 02:52:21.802618 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 02:52:21.802759 (+ 141us) server_negotiation.cc:407] Connection header received
0811 02:52:21.802864 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:52:21.802866 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:52:21.802911 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:52:21.803029 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:52:21.803553 (+ 524us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:52:21.804180 (+ 627us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:52:21.805382 (+ 1202us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:52:21.805765 (+ 383us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:52:21.805904 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:52:21.807491 (+ 1587us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:52:21.807495 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:52:21.807498 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:52:21.807526 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:52:21.809339 (+ 1813us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:52:21.809896 (+ 557us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:52:21.809900 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:52:21.809902 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:52:21.809991 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:52:21.810349 (+ 358us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:52:21.810352 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:52:21.810354 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:52:21.810578 (+ 224us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:52:21.810671 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0811 02:52:21.811035 (+ 364us) server_negotiation.cc:299] Negotiation successful
0811 02:52:21.811188 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260811 02:52:21.812390 20160 tablet_copy_service.cc:140] P 0317e693c61849e9bf42d55fbf6ef321: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer 269d7f529b994b32bc6d6d1b512371d4 ({username='oryx', principal='oryx/127.12.138.253@KRBTEST.COM'} at 127.0.0.1:54560)
I20260811 02:52:21.812553 20160 tablet_copy_service.cc:161] P 0317e693c61849e9bf42d55fbf6ef321: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer 269d7f529b994b32bc6d6d1b512371d4 at {username='oryx', principal='oryx/127.12.138.253@KRBTEST.COM'} at 127.0.0.1:54560: session id = 269d7f529b994b32bc6d6d1b512371d4-00000000000000000000000000000000
I20260811 02:52:21.814744 20160 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P 0317e693c61849e9bf42d55fbf6ef321: Tablet Copy: opened 0 blocks and 1 log segments
I20260811 02:52:21.816717 20683 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260811 02:52:21.819859 20683 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P 269d7f529b994b32bc6d6d1b512371d4: tablet copy: Starting download of 0 data blocks...
I20260811 02:52:21.820071 20683 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P 269d7f529b994b32bc6d6d1b512371d4: tablet copy: Starting download of 1 WAL segments...
I20260811 02:52:21.822029 20683 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P 269d7f529b994b32bc6d6d1b512371d4: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260811 02:52:21.823936 20160 tablet_copy_service.cc:342] P 0317e693c61849e9bf42d55fbf6ef321: Request end of tablet copy session 269d7f529b994b32bc6d6d1b512371d4-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.12.138.253@KRBTEST.COM'} at 127.0.0.1:54560
I20260811 02:52:21.824044 20160 tablet_copy_service.cc:433] P 0317e693c61849e9bf42d55fbf6ef321: ending tablet copy session 269d7f529b994b32bc6d6d1b512371d4-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer 269d7f529b994b32bc6d6d1b512371d4
I20260811 02:52:21.831753 20683 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 02:52:21.833555 20806 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:52:21.834182 20807 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:52:21.834458 20810 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:52:21.835086 20683 server_base.cc:1034] running on GCE node
I20260811 02:52:21.835284 20683 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 02:52:21.835599 20683 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 02:52:21.836732 20683 hybrid_clock.cc:648] HybridClock initialized: now 1786416741836719 us; error 32 us; skew 500 ppm
Aug 11 02:52:21 dist-test-slave-9gcw krb5kdc[20077](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416741, etypes {rep=17 tkt=17 ses=17}, oryx/127.12.138.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 02:52:21.838442 20683 init.cc:377] Logged in from keytab as oryx/127.12.138.253@KRBTEST.COM (short username oryx)
I20260811 02:52:21.838903 20683 webserver.cc:460] Webserver started at http://127.12.138.253:43799/ using document root <none> and password file <none>
I20260811 02:52:21.839098 20683 fs_manager.cc:359] Metadata directory not provided
I20260811 02:52:21.839140 20683 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 02:52:21.840626 20683 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260811 02:52:21.841672 20820 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 02:52:21.841971 20683 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260811 02:52:21.842046 20683 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786416707705494-12843-0/minicluster-data/master-1/data,/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786416707705494-12843-0/minicluster-data/master-1/wal
uuid: "269d7f529b994b32bc6d6d1b512371d4"
format_stamp: "Formatted at 2026-08-11 02:52:21 on dist-test-slave-9gcw"
server_key: "5c2eb3a03a3916185234c7b90553fb11"
server_key_iv: "c542d4c823346f3e99a039bd3f75d72a"
server_key_version: "encryptionkey@0"
I20260811 02:52:21.842128 20683 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786416707705494-12843-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786416707705494-12843-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786416707705494-12843-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 02:52:21.858155 20683 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 02:52:21.860492 20683 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 02:52:21.866672 20683 rpc_server.cc:307] RPC server started. Bound to: 127.12.138.253:42467
I20260811 02:52:21.867372 20683 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786416707705494-12843-0/minicluster-data/master-1/data/info.pb
I20260811 02:52:21.867746 20879 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.138.253:42467 every 8 connection(s)
I20260811 02:52:21.869426 20880 sys_catalog.cc:263] Verifying existing consensus state
I20260811 02:52:21.870697 20880 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 269d7f529b994b32bc6d6d1b512371d4: Bootstrap starting.
I20260811 02:52:21.881456 20880 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 269d7f529b994b32bc6d6d1b512371d4: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 02:52:21.881975 20880 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 269d7f529b994b32bc6d6d1b512371d4: Bootstrap complete.
I20260811 02:52:21.882733 20880 raft_consensus.cc:348] T 00000000000000000000000000000000 P 269d7f529b994b32bc6d6d1b512371d4 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "0317e693c61849e9bf42d55fbf6ef321" member_type: VOTER last_known_addr { host: "127.12.138.254" port: 43457 } } peers { permanent_uuid: "269d7f529b994b32bc6d6d1b512371d4" member_type: NON_VOTER last_known_addr { host: "127.12.138.253" port: 42467 } attrs { promote: true } }
I20260811 02:52:21.883023 20880 raft_consensus.cc:729] T 00000000000000000000000000000000 P 269d7f529b994b32bc6d6d1b512371d4 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 269d7f529b994b32bc6d6d1b512371d4, State: Initialized, Role: LEARNER
I20260811 02:52:21.883183 20880 consensus_queue.cc:260] T 00000000000000000000000000000000 P 269d7f529b994b32bc6d6d1b512371d4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "0317e693c61849e9bf42d55fbf6ef321" member_type: VOTER last_known_addr { host: "127.12.138.254" port: 43457 } } peers { permanent_uuid: "269d7f529b994b32bc6d6d1b512371d4" member_type: NON_VOTER last_known_addr { host: "127.12.138.253" port: 42467 } attrs { promote: true } }
I20260811 02:52:21.883659 20883 sys_catalog.cc:455] T 00000000000000000000000000000000 P 269d7f529b994b32bc6d6d1b512371d4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "0317e693c61849e9bf42d55fbf6ef321" member_type: VOTER last_known_addr { host: "127.12.138.254" port: 43457 } } peers { permanent_uuid: "269d7f529b994b32bc6d6d1b512371d4" member_type: NON_VOTER last_known_addr { host: "127.12.138.253" port: 42467 } attrs { promote: true } } }
I20260811 02:52:21.883765 20880 sys_catalog.cc:564] T 00000000000000000000000000000000 P 269d7f529b994b32bc6d6d1b512371d4 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 02:52:21.883781 20883 sys_catalog.cc:458] T 00000000000000000000000000000000 P 269d7f529b994b32bc6d6d1b512371d4 [sys.catalog]: This master's current role is: LEARNER
I20260811 02:52:21.886978 20895 catalog_manager.cc:1259] Loaded cluster ID: 101bfd3e0cf345c7b289e94b98d49ad6
I20260811 02:52:21.887039 20895 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 269d7f529b994b32bc6d6d1b512371d4: loading cluster ID for follower catalog manager: success
I20260811 02:52:21.889312 20895 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 269d7f529b994b32bc6d6d1b512371d4: acquiring CA information for follower catalog manager: success
I20260811 02:52:21.890796 20895 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 269d7f529b994b32bc6d6d1b512371d4: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260811 02:52:21.942785 20805 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 02:52:21.943146 20805 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 02:52:21.943224 20805 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 02:52:21.947520 20805 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 02:52:21.947582 20805 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 02:52:21.947602 20805 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 02:52:21.947619 20805 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 02:52:21.947634 20805 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 02:52:21.952839 20805 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.138.212:41877
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786416707705494-12843-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786416707705494-12843-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.12.138.254:43457,127.12.138.253:42467,127.12.138.252:42917
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786416707705494-12843-0/krb5kdc/oryx_127.12.138.252.keytab
--principal=oryx/127.12.138.252
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.138.252:42917
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786416707705494-12843-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.12.138.252
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20805
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786416707705494-12843-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 02:09:49 UTC on ccycloud.quasar-fsnqqc.root.comops.site
build id 4579570
I20260811 02:52:21.953944 20805 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 02:52:21.954952 20805 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 02:52:21.963495 20901 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:52:21.963501 20900 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:52:21.963501 20903 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:52:21.963681 20805 server_base.cc:1034] running on GCE node
I20260811 02:52:21.964450 20805 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 02:52:21.965632 20805 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 02:52:21.966828 20805 hybrid_clock.cc:648] HybridClock initialized: now 1786416741966795 us; error 49 us; skew 500 ppm
Aug 11 02:52:21 dist-test-slave-9gcw krb5kdc[20077](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416741, etypes {rep=17 tkt=17 ses=17}, oryx/127.12.138.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 02:52:21.969844 20805 init.cc:377] Logged in from keytab as oryx/127.12.138.252@KRBTEST.COM (short username oryx)
I20260811 02:52:21.970953 20805 webserver.cc:460] Webserver started at http://127.12.138.252:44443/ using document root <none> and password file <none>
I20260811 02:52:21.971546 20805 fs_manager.cc:359] Metadata directory not provided
I20260811 02:52:21.971612 20805 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 02:52:21.971799 20805 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 02:52:21.974145 20805 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786416707705494-12843-0/minicluster-data/master-2/data/instance:
uuid: "3b8a3f24429b46a29f1697d97b2da6d1"
format_stamp: "Formatted at 2026-08-11 02:52:21 on dist-test-slave-9gcw"
server_key: "169af108765a2c182d59563c54fa41ab"
server_key_iv: "f75faa6503ffd7bd99a64f544c79bcf2"
server_key_version: "encryptionkey@0"
I20260811 02:52:21.974766 20805 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786416707705494-12843-0/minicluster-data/master-2/wal/instance:
uuid: "3b8a3f24429b46a29f1697d97b2da6d1"
format_stamp: "Formatted at 2026-08-11 02:52:21 on dist-test-slave-9gcw"
server_key: "169af108765a2c182d59563c54fa41ab"
server_key_iv: "f75faa6503ffd7bd99a64f544c79bcf2"
server_key_version: "encryptionkey@0"
I20260811 02:52:21.978883 20805 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260811 02:52:21.981423 20911 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 02:52:21.982708 20805 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260811 02:52:21.982859 20805 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786416707705494-12843-0/minicluster-data/master-2/data,/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786416707705494-12843-0/minicluster-data/master-2/wal
uuid: "3b8a3f24429b46a29f1697d97b2da6d1"
format_stamp: "Formatted at 2026-08-11 02:52:21 on dist-test-slave-9gcw"
server_key: "169af108765a2c182d59563c54fa41ab"
server_key_iv: "f75faa6503ffd7bd99a64f544c79bcf2"
server_key_version: "encryptionkey@0"
I20260811 02:52:21.983032 20805 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786416707705494-12843-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786416707705494-12843-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786416707705494-12843-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 02:52:22.006304 20805 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 02:52:22.010413 20805 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 02:52:22.010586 20805 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 02:52:22.018002 20805 rpc_server.cc:307] RPC server started. Bound to: 127.12.138.252:42917
I20260811 02:52:22.018293 20971 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.138.252:42917 every 8 connection(s)
I20260811 02:52:22.019062 20805 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786416707705494-12843-0/minicluster-data/master-2/data/info.pb
I20260811 02:52:22.021972 20973 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 02:52:22.024173 12843 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDawEwS/build/debug/bin/kudu as pid 20805
I20260811 02:52:22.026499 20973 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.12.138.254" port: 43457 } has no permanent_uuid. Determining permanent_uuid...
I20260811 02:52:22.031765 20359 heartbeater.cc:503] Master 127.12.138.254:43457 was elected leader, sending a full tablet report...
Aug 11 02:52:22 dist-test-slave-9gcw krb5kdc[20077](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416741, etypes {rep=17 tkt=17 ses=17}, oryx/127.12.138.252@KRBTEST.COM for oryx/127.12.138.254@KRBTEST.COM
Aug 11 02:52:22 dist-test-slave-9gcw krb5kdc[20077](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416740, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.12.138.252@KRBTEST.COM
I20260811 02:52:22.040390 20182 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:52:22.028461 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54572 (local address 127.12.138.254:43457)
0811 02:52:22.028592 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0811 02:52:22.028599 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 02:52:22.028802 (+ 203us) server_negotiation.cc:407] Connection header received
0811 02:52:22.028966 (+ 164us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:52:22.028969 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:52:22.029026 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:52:22.029104 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:52:22.030005 (+ 901us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:52:22.030789 (+ 784us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:52:22.032162 (+ 1373us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:52:22.032523 (+ 361us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:52:22.032607 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:52:22.035968 (+ 3361us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:52:22.035972 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:52:22.035974 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:52:22.035997 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:52:22.037735 (+ 1738us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:52:22.038395 (+ 660us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:52:22.038400 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:52:22.038402 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:52:22.038489 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:52:22.038926 (+ 437us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:52:22.038930 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:52:22.038932 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:52:22.039194 (+ 262us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:52:22.039270 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0811 02:52:22.040021 (+ 751us) server_negotiation.cc:299] Negotiation successful
0811 02:52:22.040179 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260811 02:52:22.041520 20976 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:52:22.028256 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.138.254:43457 (local address 127.0.0.1:54572)
0811 02:52:22.028676 (+ 420us) negotiation.cc:107] Waiting for socket to connect
0811 02:52:22.028715 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0811 02:52:22.028850 (+ 135us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:52:22.029216 (+ 366us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:52:22.029222 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:52:22.029670 (+ 448us) client_negotiation.cc:189] Negotiated authn=SASL
0811 02:52:22.029844 (+ 174us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:52:22.029854 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:52:22.030932 (+ 1078us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:52:22.030938 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:52:22.032014 (+ 1076us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:52:22.032026 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:52:22.032859 (+ 833us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:52:22.032864 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:52:22.033056 (+ 192us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:52:22.033754 (+ 698us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 02:52:22.033813 (+ 59us) client_negotiation.cc:656] calling sasl_client_start()
0811 02:52:22.035681 (+ 1868us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 02:52:22.037854 (+ 2173us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:52:22.037865 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:52:22.037877 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:52:22.038289 (+ 412us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:52:22.038589 (+ 300us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:52:22.038592 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:52:22.038593 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:52:22.038816 (+ 223us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:52:22.039306 (+ 490us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 02:52:22.039313 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 02:52:22.039762 (+ 449us) client_negotiation.cc:769] Sending connection context
0811 02:52:22.039963 (+ 201us) client_negotiation.cc:240] Negotiation successful
0811 02:52:22.040384 (+ 421us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":236,"mutex_wait_us":107,"thread_start_us":144,"threads_started":1}
I20260811 02:52:22.041520 20975 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:52:22.026523 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33438 (local address 127.12.138.252:42917)
0811 02:52:22.027035 (+ 512us) server_negotiation.cc:206] Beginning negotiation
0811 02:52:22.027051 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0811 02:52:22.027089 (+ 38us) server_negotiation.cc:407] Connection header received
0811 02:52:22.027738 (+ 649us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:52:22.027756 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:52:22.028178 (+ 422us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:52:22.029077 (+ 899us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:52:22.029547 (+ 470us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:52:22.030899 (+ 1352us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:52:22.032079 (+ 1180us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:52:22.032641 (+ 562us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:52:22.033181 (+ 540us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:52:22.036364 (+ 3183us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:52:22.036370 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:52:22.036383 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:52:22.036406 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:52:22.038196 (+ 1790us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:52:22.038740 (+ 544us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:52:22.038746 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:52:22.038748 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:52:22.038824 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:52:22.039121 (+ 297us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:52:22.039123 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:52:22.039124 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:52:22.039761 (+ 637us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:52:22.039962 (+ 201us) server_negotiation.cc:1035] Waiting for connection context
0811 02:52:22.040203 (+ 241us) server_negotiation.cc:299] Negotiation successful
0811 02:52:22.040434 (+ 231us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":128,"server-negotiator.queue_time_us":326,"thread_start_us":155,"threads_started":1}
I20260811 02:52:22.044179 20973 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.12.138.253" port: 42467 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 02:52:22 dist-test-slave-9gcw krb5kdc[20077](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416741, etypes {rep=17 tkt=17 ses=17}, oryx/127.12.138.252@KRBTEST.COM for oryx/127.12.138.253@KRBTEST.COM
I20260811 02:52:22.052618 20976 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:52:22.044749 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.138.253:42467 (local address 127.0.0.1:60376)
0811 02:52:22.044887 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0811 02:52:22.044903 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 02:52:22.044990 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:52:22.045447 (+ 457us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:52:22.045451 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:52:22.045631 (+ 180us) client_negotiation.cc:189] Negotiated authn=SASL
0811 02:52:22.045736 (+ 105us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:52:22.045741 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:52:22.046649 (+ 908us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:52:22.046651 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:52:22.047176 (+ 525us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:52:22.047184 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:52:22.047643 (+ 459us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:52:22.047646 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:52:22.047718 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:52:22.048148 (+ 430us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 02:52:22.048159 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0811 02:52:22.049473 (+ 1314us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 02:52:22.051094 (+ 1621us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:52:22.051099 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:52:22.051101 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:52:22.051304 (+ 203us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:52:22.051599 (+ 295us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:52:22.051601 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:52:22.051603 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:52:22.051684 (+ 81us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:52:22.052070 (+ 386us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 02:52:22.052073 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 02:52:22.052175 (+ 102us) client_negotiation.cc:769] Sending connection context
0811 02:52:22.052288 (+ 113us) client_negotiation.cc:240] Negotiation successful
0811 02:52:22.052438 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":55}
I20260811 02:52:22.052636 20977 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:52:22.044850 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60376 (local address 127.12.138.253:42467)
0811 02:52:22.045124 (+ 274us) server_negotiation.cc:206] Beginning negotiation
0811 02:52:22.045128 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 02:52:22.045225 (+ 97us) server_negotiation.cc:407] Connection header received
0811 02:52:22.045282 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:52:22.045285 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:52:22.045327 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:52:22.045417 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:52:22.045850 (+ 433us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:52:22.046552 (+ 702us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:52:22.047300 (+ 748us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:52:22.047535 (+ 235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:52:22.047627 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:52:22.049616 (+ 1989us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:52:22.049619 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:52:22.049621 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:52:22.049643 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:52:22.050976 (+ 1333us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:52:22.051403 (+ 427us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:52:22.051408 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:52:22.051410 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:52:22.051493 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:52:22.051775 (+ 282us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:52:22.051780 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:52:22.051782 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:52:22.051970 (+ 188us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:52:22.052083 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0811 02:52:22.052361 (+ 278us) server_negotiation.cc:299] Negotiation successful
0811 02:52:22.052487 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":208,"thread_start_us":117,"threads_started":1}
I20260811 02:52:22.053524 20973 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.12.138.252" port: 42917 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 02:52:22 dist-test-slave-9gcw krb5kdc[20077](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416741, etypes {rep=17 tkt=17 ses=17}, oryx/127.12.138.252@KRBTEST.COM for oryx/127.12.138.252@KRBTEST.COM
I20260811 02:52:22.060652 20976 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:52:22.053992 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.138.252:42917 (local address 127.0.0.1:33446)
0811 02:52:22.054109 (+ 117us) negotiation.cc:107] Waiting for socket to connect
0811 02:52:22.054124 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 02:52:22.054197 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:52:22.054383 (+ 186us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:52:22.054386 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:52:22.054511 (+ 125us) client_negotiation.cc:189] Negotiated authn=SASL
0811 02:52:22.054582 (+ 71us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:52:22.054586 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:52:22.055252 (+ 666us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:52:22.055254 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:52:22.055748 (+ 494us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:52:22.055755 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:52:22.056153 (+ 398us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:52:22.056155 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:52:22.056211 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:52:22.056609 (+ 398us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 02:52:22.056620 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0811 02:52:22.057915 (+ 1295us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 02:52:22.059391 (+ 1476us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:52:22.059393 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:52:22.059395 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:52:22.059581 (+ 186us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:52:22.059819 (+ 238us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:52:22.059821 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:52:22.059822 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:52:22.059894 (+ 72us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:52:22.060220 (+ 326us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 02:52:22.060222 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 02:52:22.060302 (+ 80us) client_negotiation.cc:769] Sending connection context
0811 02:52:22.060381 (+ 79us) client_negotiation.cc:240] Negotiation successful
0811 02:52:22.060481 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":48}
I20260811 02:52:22.060657 20975 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:52:22.054078 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33446 (local address 127.12.138.252:42917)
0811 02:52:22.054189 (+ 111us) server_negotiation.cc:206] Beginning negotiation
0811 02:52:22.054192 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 02:52:22.054203 (+ 11us) server_negotiation.cc:407] Connection header received
0811 02:52:22.054255 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:52:22.054257 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:52:22.054291 (+ 34us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:52:22.054355 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:52:22.054675 (+ 320us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:52:22.055161 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:52:22.055841 (+ 680us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:52:22.056061 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:52:22.056137 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:52:22.058060 (+ 1923us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:52:22.058063 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:52:22.058065 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:52:22.058086 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:52:22.059294 (+ 1208us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:52:22.059664 (+ 370us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:52:22.059666 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:52:22.059668 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:52:22.059738 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:52:22.059969 (+ 231us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:52:22.059971 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:52:22.059972 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:52:22.060142 (+ 170us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:52:22.060215 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0811 02:52:22.060440 (+ 225us) server_negotiation.cc:299] Negotiation successful
0811 02:52:22.060532 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260811 02:52:22.064021 20973 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3b8a3f24429b46a29f1697d97b2da6d1: Bootstrap starting.
I20260811 02:52:22.066339 20973 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3b8a3f24429b46a29f1697d97b2da6d1: Neither blocks nor log segments found. Creating new log.
I20260811 02:52:22.067022 20973 log.cc:824] T 00000000000000000000000000000000 P 3b8a3f24429b46a29f1697d97b2da6d1: Log is configured to *not* fsync() on all Append() calls
I20260811 02:52:22.068886 20973 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3b8a3f24429b46a29f1697d97b2da6d1: No bootstrap required, opened a new log
I20260811 02:52:22.071848 20973 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3b8a3f24429b46a29f1697d97b2da6d1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0317e693c61849e9bf42d55fbf6ef321" member_type: VOTER last_known_addr { host: "127.12.138.254" port: 43457 } } peers { permanent_uuid: "269d7f529b994b32bc6d6d1b512371d4" member_type: VOTER last_known_addr { host: "127.12.138.253" port: 42467 } } peers { permanent_uuid: "3b8a3f24429b46a29f1697d97b2da6d1" member_type: VOTER last_known_addr { host: "127.12.138.252" port: 42917 } }
I20260811 02:52:22.072052 20973 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3b8a3f24429b46a29f1697d97b2da6d1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 02:52:22.072086 20973 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3b8a3f24429b46a29f1697d97b2da6d1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3b8a3f24429b46a29f1697d97b2da6d1, State: Initialized, Role: FOLLOWER
I20260811 02:52:22.072499 20973 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3b8a3f24429b46a29f1697d97b2da6d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0317e693c61849e9bf42d55fbf6ef321" member_type: VOTER last_known_addr { host: "127.12.138.254" port: 43457 } } peers { permanent_uuid: "269d7f529b994b32bc6d6d1b512371d4" member_type: VOTER last_known_addr { host: "127.12.138.253" port: 42467 } } peers { permanent_uuid: "3b8a3f24429b46a29f1697d97b2da6d1" member_type: VOTER last_known_addr { host: "127.12.138.252" port: 42917 } }
I20260811 02:52:22.072953 20979 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3b8a3f24429b46a29f1697d97b2da6d1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0317e693c61849e9bf42d55fbf6ef321" member_type: VOTER last_known_addr { host: "127.12.138.254" port: 43457 } } peers { permanent_uuid: "269d7f529b994b32bc6d6d1b512371d4" member_type: VOTER last_known_addr { host: "127.12.138.253" port: 42467 } } peers { permanent_uuid: "3b8a3f24429b46a29f1697d97b2da6d1" member_type: VOTER last_known_addr { host: "127.12.138.252" port: 42917 } } }
I20260811 02:52:22.073096 20979 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3b8a3f24429b46a29f1697d97b2da6d1 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 02:52:22.073675 20973 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3b8a3f24429b46a29f1697d97b2da6d1 [sys.catalog]: configured and running, proceeding with master startup.
W20260811 02:52:22.078644 20993 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 3b8a3f24429b46a29f1697d97b2da6d1: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 02:52:22.078752 20993 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 02:52:22.082134 20805 master_runner.cc:428] Detected that this master 3b8a3f24429b46a29f1697d97b2da6d1 is joining an existing cluster
I20260811 02:52:22.082211 20805 master_runner.cc:432] Initiating AddMaster RPC to add 127.12.138.252:42917
I20260811 02:52:22.093675 20182 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:52:22.084901 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54582 (local address 127.12.138.254:43457)
0811 02:52:22.085066 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0811 02:52:22.085070 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 02:52:22.085085 (+ 15us) server_negotiation.cc:407] Connection header received
0811 02:52:22.085208 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:52:22.085211 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:52:22.085270 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:52:22.085377 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:52:22.085966 (+ 589us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:52:22.086855 (+ 889us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:52:22.088295 (+ 1440us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:52:22.088636 (+ 341us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:52:22.088709 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:52:22.090294 (+ 1585us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:52:22.090298 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:52:22.090301 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:52:22.090327 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:52:22.091861 (+ 1534us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:52:22.092321 (+ 460us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:52:22.092324 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:52:22.092326 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:52:22.092416 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:52:22.092706 (+ 290us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:52:22.092710 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:52:22.092712 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:52:22.092911 (+ 199us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:52:22.093025 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0811 02:52:22.093389 (+ 364us) server_negotiation.cc:299] Negotiation successful
0811 02:52:22.093509 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":83}
I20260811 02:52:22.093693 21000 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:52:22.084404 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.138.254:43457 (local address 127.0.0.1:54582)
0811 02:52:22.084678 (+ 274us) negotiation.cc:107] Waiting for socket to connect
0811 02:52:22.084697 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0811 02:52:22.084809 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:52:22.085397 (+ 588us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:52:22.085400 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:52:22.085655 (+ 255us) client_negotiation.cc:189] Negotiated authn=SASL
0811 02:52:22.085827 (+ 172us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:52:22.085837 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:52:22.087132 (+ 1295us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:52:22.087135 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:52:22.088164 (+ 1029us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:52:22.088175 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:52:22.088896 (+ 721us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:52:22.088899 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:52:22.089111 (+ 212us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:52:22.089724 (+ 613us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 02:52:22.089737 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0811 02:52:22.090132 (+ 395us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 02:52:22.091984 (+ 1852us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:52:22.091989 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:52:22.091992 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:52:22.092211 (+ 219us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:52:22.092513 (+ 302us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:52:22.092517 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:52:22.092518 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:52:22.092611 (+ 93us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:52:22.093015 (+ 404us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 02:52:22.093018 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 02:52:22.093183 (+ 165us) client_negotiation.cc:769] Sending connection context
0811 02:52:22.093318 (+ 135us) client_negotiation.cc:240] Negotiation successful
0811 02:52:22.093490 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":184,"mutex_wait_us":48,"thread_start_us":104,"threads_started":1}
I20260811 02:52:22.095464 12843 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDawEwS/build/debug/bin/kudu with pid 20207
I20260811 02:52:22.104110 20979 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3b8a3f24429b46a29f1697d97b2da6d1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 02:52:22.104326 20979 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3b8a3f24429b46a29f1697d97b2da6d1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0317e693c61849e9bf42d55fbf6ef321" member_type: VOTER last_known_addr { host: "127.12.138.254" port: 43457 } } peers { permanent_uuid: "269d7f529b994b32bc6d6d1b512371d4" member_type: VOTER last_known_addr { host: "127.12.138.253" port: 42467 } } peers { permanent_uuid: "3b8a3f24429b46a29f1697d97b2da6d1" member_type: VOTER last_known_addr { host: "127.12.138.252" port: 42917 } }
I20260811 02:52:22.105558 20979 leader_election.cc:288] T 00000000000000000000000000000000 P 3b8a3f24429b46a29f1697d97b2da6d1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0317e693c61849e9bf42d55fbf6ef321 (127.12.138.254:43457), 269d7f529b994b32bc6d6d1b512371d4 (127.12.138.253:42467)
I20260811 02:52:22.106809 20143 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "3b8a3f24429b46a29f1697d97b2da6d1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0317e693c61849e9bf42d55fbf6ef321" is_pre_election: true
I20260811 02:52:22.107434 20851 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "3b8a3f24429b46a29f1697d97b2da6d1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "269d7f529b994b32bc6d6d1b512371d4" is_pre_election: true
I20260811 02:52:22.107621 20851 raft_consensus.cc:1799] T 00000000000000000000000000000000 P 269d7f529b994b32bc6d6d1b512371d4 [term 1 LEARNER]: Handling vote request from an unknown peer 3b8a3f24429b46a29f1697d97b2da6d1
I20260811 02:52:22.107721 20851 raft_consensus.cc:2396] T 00000000000000000000000000000000 P 269d7f529b994b32bc6d6d1b512371d4 [term 1 LEARNER]: Leader pre-election vote request: Denying vote to candidate 3b8a3f24429b46a29f1697d97b2da6d1 for term 1 because replica has last-logged OpId of term: 1 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 0 index: 0.
I20260811 02:52:22.108146 20914 leader_election.cc:302] T 00000000000000000000000000000000 P 3b8a3f24429b46a29f1697d97b2da6d1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b8a3f24429b46a29f1697d97b2da6d1; no voters: 0317e693c61849e9bf42d55fbf6ef321, 269d7f529b994b32bc6d6d1b512371d4
I20260811 02:52:22.108901 20979 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3b8a3f24429b46a29f1697d97b2da6d1 [term 0 FOLLOWER]: Advancing to term 1
I20260811 02:52:22.110401 20979 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 3b8a3f24429b46a29f1697d97b2da6d1 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260811 02:52:22.113744 12843 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDawEwS/build/debug/bin/kudu
/tmp/dist-test-taskDawEwS/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786416707705494-12843-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786416707705494-12843-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786416707705494-12843-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786416707705494-12843-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.138.193:41461
--local_ip_for_outbound_sockets=127.12.138.193
--tserver_master_addrs=127.12.138.254:43457
--webserver_port=35107
--webserver_interface=127.12.138.193
--keytab_file=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786416707705494-12843-0/krb5kdc/oryx_127.12.138.193.keytab
--principal=oryx/127.12.138.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.138.212:41877
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.12.138.254:43457,127.12.138.253:42467
--tserver_master_addrs=127.12.138.254:43457,127.12.138.253:42467,127.12.138.252:42917 with env {KRB5CCNAME=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786416707705494-12843-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786416707705494-12843-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786416707705494-12843-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 11 02:52:22 dist-test-slave-9gcw krb5kdc[20077](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416740, etypes {rep=17 tkt=17 ses=17}, oryx/127.12.138.254@KRBTEST.COM for oryx/127.12.138.252@KRBTEST.COM
I20260811 02:52:22.119940 20977 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:52:22.084874 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60386 (local address 127.12.138.253:42467)
0811 02:52:22.085015 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0811 02:52:22.085021 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 02:52:22.085702 (+ 681us) server_negotiation.cc:407] Connection header received
0811 02:52:22.085887 (+ 185us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:52:22.085891 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:52:22.085942 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:52:22.086023 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:52:22.086675 (+ 652us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:52:22.087503 (+ 828us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:52:22.088882 (+ 1379us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:52:22.089278 (+ 396us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:52:22.089357 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:52:22.115124 (+ 25767us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:52:22.115128 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:52:22.115131 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:52:22.115163 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:52:22.117991 (+ 2828us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:52:22.118435 (+ 444us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:52:22.118439 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:52:22.118441 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:52:22.118524 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:52:22.118824 (+ 300us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:52:22.118828 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:52:22.118830 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:52:22.119096 (+ 266us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:52:22.119192 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0811 02:52:22.119562 (+ 370us) server_negotiation.cc:299] Negotiation successful
0811 02:52:22.119760 (+ 198us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260811 02:52:22.120635 21002 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:52:22.084477 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.138.253:42467 (local address 127.0.0.1:60386)
0811 02:52:22.085611 (+ 1134us) negotiation.cc:107] Waiting for socket to connect
0811 02:52:22.085627 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 02:52:22.085762 (+ 135us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:52:22.086174 (+ 412us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:52:22.086177 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:52:22.086406 (+ 229us) client_negotiation.cc:189] Negotiated authn=SASL
0811 02:52:22.086533 (+ 127us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:52:22.086541 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:52:22.087754 (+ 1213us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:52:22.087757 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:52:22.088730 (+ 973us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:52:22.088742 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:52:22.089512 (+ 770us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:52:22.089515 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:52:22.089609 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:52:22.114330 (+ 24721us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 02:52:22.114344 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0811 02:52:22.114726 (+ 382us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 02:52:22.118094 (+ 3368us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:52:22.118097 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:52:22.118100 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:52:22.118326 (+ 226us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:52:22.118630 (+ 304us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:52:22.118633 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:52:22.118635 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:52:22.118713 (+ 78us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:52:22.119238 (+ 525us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 02:52:22.119244 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 02:52:22.119352 (+ 108us) client_negotiation.cc:769] Sending connection context
0811 02:52:22.120177 (+ 825us) client_negotiation.cc:240] Negotiation successful
0811 02:52:22.120388 (+ 211us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1073,"mutex_wait_us":43,"thread_start_us":144,"threads_started":1}
I20260811 02:52:22.122730 20975 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:52:22.085189 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33448 (local address 127.12.138.252:42917)
0811 02:52:22.085294 (+ 105us) server_negotiation.cc:206] Beginning negotiation
0811 02:52:22.085299 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 02:52:22.086033 (+ 734us) server_negotiation.cc:407] Connection header received
0811 02:52:22.086157 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:52:22.086160 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:52:22.086193 (+ 33us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:52:22.086234 (+ 41us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:52:22.086781 (+ 547us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:52:22.087497 (+ 716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:52:22.088651 (+ 1154us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:52:22.088941 (+ 290us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:52:22.089037 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:52:22.115359 (+ 26322us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:52:22.115366 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:52:22.115370 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:52:22.115403 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:52:22.117840 (+ 2437us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:52:22.121606 (+ 3766us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:52:22.121609 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:52:22.121611 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:52:22.121694 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:52:22.121957 (+ 263us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:52:22.121960 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:52:22.121962 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:52:22.122138 (+ 176us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:52:22.122213 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0811 02:52:22.122467 (+ 254us) server_negotiation.cc:299] Negotiation successful
0811 02:52:22.122573 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":16,"server-negotiator.queue_time_us":43}
I20260811 02:52:22.123929 20788 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:52:22.110375 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.138.252:42917 (local address 127.0.0.1:33458)
0811 02:52:22.110647 (+ 272us) negotiation.cc:107] Waiting for socket to connect
0811 02:52:22.110659 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 02:52:22.110735 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:52:22.112099 (+ 1364us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:52:22.112102 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:52:22.112331 (+ 229us) client_negotiation.cc:189] Negotiated authn=SASL
0811 02:52:22.112427 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:52:22.112433 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:52:22.114107 (+ 1674us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:52:22.114113 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:52:22.115175 (+ 1062us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:52:22.115192 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:52:22.116626 (+ 1434us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:52:22.116631 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:52:22.116747 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:52:22.117190 (+ 443us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 02:52:22.117209 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0811 02:52:22.118969 (+ 1760us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 02:52:22.122134 (+ 3165us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:52:22.122137 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:52:22.122139 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:52:22.122342 (+ 203us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:52:22.122628 (+ 286us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:52:22.122634 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:52:22.122636 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:52:22.122733 (+ 97us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:52:22.123379 (+ 646us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 02:52:22.123382 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 02:52:22.123480 (+ 98us) client_negotiation.cc:769] Sending connection context
0811 02:52:22.123588 (+ 108us) client_negotiation.cc:240] Negotiation successful
0811 02:52:22.123768 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":183}
I20260811 02:52:22.123946 21003 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:52:22.110855 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33458 (local address 127.12.138.252:42917)
0811 02:52:22.111138 (+ 283us) server_negotiation.cc:206] Beginning negotiation
0811 02:52:22.111147 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 02:52:22.111169 (+ 22us) server_negotiation.cc:407] Connection header received
0811 02:52:22.111230 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:52:22.111232 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:52:22.111278 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:52:22.112084 (+ 806us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:52:22.112556 (+ 472us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:52:22.113960 (+ 1404us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:52:22.116027 (+ 2067us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:52:22.116447 (+ 420us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:52:22.116967 (+ 520us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:52:22.119158 (+ 2191us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:52:22.119164 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:52:22.119167 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:52:22.119189 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:52:22.122007 (+ 2818us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:52:22.122445 (+ 438us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:52:22.122450 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:52:22.122452 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:52:22.122534 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:52:22.123021 (+ 487us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:52:22.123025 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:52:22.123027 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:52:22.123245 (+ 218us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:52:22.123334 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0811 02:52:22.123690 (+ 356us) server_negotiation.cc:299] Negotiation successful
0811 02:52:22.123802 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":214,"thread_start_us":133,"threads_started":1}
I20260811 02:52:22.125298 21001 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:52:22.085258 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.138.252:42917 (local address 127.0.0.1:33448)
0811 02:52:22.085966 (+ 708us) negotiation.cc:107] Waiting for socket to connect
0811 02:52:22.085979 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 02:52:22.086077 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:52:22.086366 (+ 289us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:52:22.086368 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:52:22.086575 (+ 207us) client_negotiation.cc:189] Negotiated authn=SASL
0811 02:52:22.086676 (+ 101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:52:22.086683 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:52:22.087632 (+ 949us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:52:22.087635 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:52:22.088545 (+ 910us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:52:22.088554 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:52:22.089065 (+ 511us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:52:22.089068 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:52:22.089173 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:52:22.113673 (+ 24500us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 02:52:22.113691 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0811 02:52:22.114167 (+ 476us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 02:52:22.120776 (+ 6609us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:52:22.120779 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:52:22.120781 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:52:22.121030 (+ 249us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:52:22.121786 (+ 756us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:52:22.121788 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:52:22.121790 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:52:22.121876 (+ 86us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:52:22.122253 (+ 377us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 02:52:22.122256 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 02:52:22.122326 (+ 70us) client_negotiation.cc:769] Sending connection context
0811 02:52:22.122852 (+ 526us) client_negotiation.cc:240] Negotiation successful
0811 02:52:22.122977 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":644,"mutex_wait_us":45,"thread_start_us":103,"threads_started":1}
I20260811 02:52:22.125268 20136 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.12.138.252:42917: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "3b8a3f24429b46a29f1697d97b2da6d1"
member_type: NON_VOTER
last_known_addr {
host: "127.12.138.252"
port: 42917
}
attrs {
promote: true
}
}
dest_uuid: "0317e693c61849e9bf42d55fbf6ef321"
cas_config_opid_index: 5
I20260811 02:52:22.125916 20136 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0317e693c61849e9bf42d55fbf6ef321 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "0317e693c61849e9bf42d55fbf6ef321" member_type: VOTER last_known_addr { host: "127.12.138.254" port: 43457 } } peers { permanent_uuid: "269d7f529b994b32bc6d6d1b512371d4" member_type: NON_VOTER last_known_addr { host: "127.12.138.253" port: 42467 } attrs { promote: true } } peers { permanent_uuid: "3b8a3f24429b46a29f1697d97b2da6d1" member_type: NON_VOTER last_known_addr { host: "127.12.138.252" port: 42917 } attrs { promote: true } }
I20260811 02:52:22.127051 20792 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 0317e693c61849e9bf42d55fbf6ef321 [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER 3b8a3f24429b46a29f1697d97b2da6d1 (127.12.138.252) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "0317e693c61849e9bf42d55fbf6ef321" member_type: VOTER last_known_addr { host: "127.12.138.254" port: 43457 } } peers { permanent_uuid: "269d7f529b994b32bc6d6d1b512371d4" member_type: NON_VOTER last_known_addr { host: "127.12.138.253" port: 42467 } attrs { promote: true } } peers { permanent_uuid: "3b8a3f24429b46a29f1697d97b2da6d1" member_type: NON_VOTER last_known_addr { host: "127.12.138.252" port: 42917 } attrs { promote: true } } }
I20260811 02:52:22.128393 20943 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 3b8a3f24429b46a29f1697d97b2da6d1 [term 1 FOLLOWER]: Refusing update from remote peer 0317e693c61849e9bf42d55fbf6ef321: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260811 02:52:22.128535 20792 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.12.138.252:42917
I20260811 02:52:22.128719 20792 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0317e693c61849e9bf42d55fbf6ef321 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "0317e693c61849e9bf42d55fbf6ef321" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "0317e693c61849e9bf42d55fbf6ef321" member_type: VOTER last_known_addr { host: "127.12.138.254" port: 43457 } } peers { permanent_uuid: "269d7f529b994b32bc6d6d1b512371d4" member_type: NON_VOTER last_known_addr { host: "127.12.138.253" port: 42467 } attrs { promote: true } } peers { permanent_uuid: "3b8a3f24429b46a29f1697d97b2da6d1" member_type: NON_VOTER last_known_addr { host: "127.12.138.252" port: 42917 } attrs { promote: true } } }
I20260811 02:52:22.128834 20792 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0317e693c61849e9bf42d55fbf6ef321 [sys.catalog]: This master's current role is: LEADER
I20260811 02:52:22.129139 20791 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 0317e693c61849e9bf42d55fbf6ef321 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b8a3f24429b46a29f1697d97b2da6d1" member_type: NON_VOTER last_known_addr { host: "127.12.138.252" port: 42917 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 02:52:22.130285 20805 master.cc:537] Master@127.12.138.252:42917 shutting down...
I20260811 02:52:22.133347 20943 raft_consensus.cc:858] T 00000000000000000000000000000000 P 3b8a3f24429b46a29f1697d97b2da6d1 [term 1 NON_PARTICIPANT]: Illegal state: RaftConfig change currently pending. Only one is allowed at a time.
Committed config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0317e693c61849e9bf42d55fbf6ef321" member_type: VOTER last_known_addr { host: "127.12.138.254" port: 43457 } } peers { permanent_uuid: "269d7f529b994b32bc6d6d1b512371d4" member_type: VOTER last_known_addr { host: "127.12.138.253" port: 42467 } } peers { permanent_uuid: "3b8a3f24429b46a29f1697d97b2da6d1" member_type: VOTER last_known_addr { host: "127.12.138.252" port: 42917 } }.
Pending config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "0317e693c61849e9bf42d55fbf6ef321" member_type: VOTER last_known_addr { host: "127.12.138.254" port: 43457 } } peers { permanent_uuid: "269d7f529b994b32bc6d6d1b512371d4" member_type: NON_VOTER last_known_addr { host: "127.12.138.253" port: 42467 } attrs { promote: true } }:
New config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "0317e693c61849e9bf42d55fbf6ef321" member_type: VOTER last_known_addr { host: "127.12.138.254" port: 43457 } } peers { permanent_uuid: "269d7f529b994b32bc6d6d1b512371d4" member_type: NON_VOTER last_known_addr { host: "127.12.138.253" port: 42467 } attrs { promote: true } } peers { permanent_uuid: "3b8a3f24429b46a29f1697d97b2da6d1" member_type: NON_VOTER last_known_addr { host: "127.12.138.252" port: 42917 } attrs { promote: true } }
W20260811 02:52:22.133472 20943 raft_consensus.cc:1565] T 00000000000000000000000000000000 P 3b8a3f24429b46a29f1697d97b2da6d1 [term 1 NON_PARTICIPANT]: Could not prepare op 'term: 1 index: 6' and following 0 ops. Status for this op: Illegal state: RaftConfig change currently pending. Only one is allowed at a time.
Committed config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0317e693c61849e9bf42d55fbf6ef321" member_type: VOTER last_known_addr { host: "127.12.138.254" port: 43457 } } peers { permanent_uuid: "269d7f529b994b32bc6d6d1b512371d4" member_type: VOTER last_known_addr { host: "127.12.138.253" port: 42467 } } peers { permanent_uuid: "3b8a3f24429b46a29f1697d97b2da6d1" member_type: VOTER last_known_addr { host: "127.12.138.252" port: 42917 } }.
Pending config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "0317e693c61849e9bf42d55fbf6ef321" member_type: VOTER last_known_addr { host: "127.12.138.254" port: 43457 } } peers { permanent_uuid: "269d7f529b994b32bc6d6d1b512371d4" member_type: NON_VOTER last_known_addr { host: "127.12.138.253" port: 42467 } attrs { promote: true } }:
New config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "0317e693c61849e9bf42d55fbf6ef321" member_type: VOTER last_known_addr { host: "127.12.138.254" port: 43457 } } peers { permanent_uuid: "269d7f529b994b32bc6d6d1b512371d4" member_type: NON_VOTER last_known_addr { host: "127.12.138.253" port: 42467 } attrs { promote: true } } peers { permanent_uuid: "3b8a3f24429b46a29f1697d97b2da6d1" member_type: NON_VOTER last_known_addr { host: "127.12.138.252" port: 42917 } attrs { promote: true } }
I20260811 02:52:22.135121 20943 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 3b8a3f24429b46a29f1697d97b2da6d1 [term 1 NON_PARTICIPANT]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER 269d7f529b994b32bc6d6d1b512371d4 (127.12.138.253) added. New config: { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "0317e693c61849e9bf42d55fbf6ef321" member_type: VOTER last_known_addr { host: "127.12.138.254" port: 43457 } } peers { permanent_uuid: "269d7f529b994b32bc6d6d1b512371d4" member_type: NON_VOTER last_known_addr { host: "127.12.138.253" port: 42467 } attrs { promote: true } } }
I20260811 02:52:22.136197 21005 mvcc.cc:205] Tried to move back new op lower bound from 7317162970237308928 to 7317162970083065856. Current Snapshot: MvccSnapshot[applied={T|T < 7317162970118369280}]
I20260811 02:52:22.137586 20979 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3b8a3f24429b46a29f1697d97b2da6d1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0317e693c61849e9bf42d55fbf6ef321. Latest consensus state: current_term: 1 leader_uuid: "0317e693c61849e9bf42d55fbf6ef321" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "0317e693c61849e9bf42d55fbf6ef321" member_type: VOTER last_known_addr { host: "127.12.138.254" port: 43457 } } peers { permanent_uuid: "269d7f529b994b32bc6d6d1b512371d4" member_type: NON_VOTER last_known_addr { host: "127.12.138.253" port: 42467 } attrs { promote: true } } }
I20260811 02:52:22.137754 20979 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3b8a3f24429b46a29f1697d97b2da6d1 [sys.catalog]: This master's current role is: NON_PARTICIPANT
I20260811 02:52:22.137861 20979 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3b8a3f24429b46a29f1697d97b2da6d1 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "0317e693c61849e9bf42d55fbf6ef321" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "0317e693c61849e9bf42d55fbf6ef321" member_type: VOTER last_known_addr { host: "127.12.138.254" port: 43457 } } peers { permanent_uuid: "269d7f529b994b32bc6d6d1b512371d4" member_type: NON_VOTER last_known_addr { host: "127.12.138.253" port: 42467 } attrs { promote: true } } }
I20260811 02:52:22.137936 20979 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3b8a3f24429b46a29f1697d97b2da6d1 [sys.catalog]: This master's current role is: NON_PARTICIPANT
I20260811 02:52:22.138025 20979 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3b8a3f24429b46a29f1697d97b2da6d1 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "0317e693c61849e9bf42d55fbf6ef321" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "0317e693c61849e9bf42d55fbf6ef321" member_type: VOTER last_known_addr { host: "127.12.138.254" port: 43457 } } peers { permanent_uuid: "269d7f529b994b32bc6d6d1b512371d4" member_type: NON_VOTER last_known_addr { host: "127.12.138.253" port: 42467 } attrs { promote: true } } }
I20260811 02:52:22.138113 20979 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3b8a3f24429b46a29f1697d97b2da6d1 [sys.catalog]: This master's current role is: NON_PARTICIPANT
I20260811 02:52:22.141211 20977 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:52:22.127483 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60398 (local address 127.12.138.253:42467)
0811 02:52:22.128844 (+ 1361us) server_negotiation.cc:206] Beginning negotiation
0811 02:52:22.128849 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 02:52:22.129240 (+ 391us) server_negotiation.cc:407] Connection header received
0811 02:52:22.129311 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:52:22.129316 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:52:22.129379 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:52:22.129478 (+ 99us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 02:52:22.129846 (+ 368us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:52:22.130663 (+ 817us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:52:22.139478 (+ 8815us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:52:22.140164 (+ 686us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:52:22.140260 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:52:22.140449 (+ 189us) server_negotiation.cc:1035] Waiting for connection context
0811 02:52:22.140988 (+ 539us) server_negotiation.cc:299] Negotiation successful
0811 02:52:22.141065 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1268}
I20260811 02:52:22.141218 20788 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:52:22.127243 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.138.253:42467 (local address 127.0.0.1:60398)
0811 02:52:22.128943 (+ 1700us) negotiation.cc:107] Waiting for socket to connect
0811 02:52:22.128962 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0811 02:52:22.129092 (+ 130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:52:22.129643 (+ 551us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:52:22.129646 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:52:22.129661 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 02:52:22.129733 (+ 72us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:52:22.129737 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:52:22.137959 (+ 8222us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:52:22.137964 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:52:22.139261 (+ 1297us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:52:22.139279 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:52:22.140711 (+ 1432us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:52:22.140716 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:52:22.140826 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:52:22.140863 (+ 37us) client_negotiation.cc:769] Sending connection context
0811 02:52:22.140955 (+ 92us) client_negotiation.cc:240] Negotiation successful
0811 02:52:22.141047 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1627}
I20260811 02:52:22.142146 20851 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 269d7f529b994b32bc6d6d1b512371d4 [term 1 LEARNER]: Refusing update from remote peer 0317e693c61849e9bf42d55fbf6ef321: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260811 02:52:22.142771 20792 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 0317e693c61849e9bf42d55fbf6ef321 [LEADER]: Connected to new peer: Peer: permanent_uuid: "269d7f529b994b32bc6d6d1b512371d4" member_type: NON_VOTER last_known_addr { host: "127.12.138.253" port: 42467 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260811 02:52:22.144187 20791 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0317e693c61849e9bf42d55fbf6ef321 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "0317e693c61849e9bf42d55fbf6ef321" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "0317e693c61849e9bf42d55fbf6ef321" member_type: VOTER last_known_addr { host: "127.12.138.254" port: 43457 } } peers { permanent_uuid: "269d7f529b994b32bc6d6d1b512371d4" member_type: NON_VOTER last_known_addr { host: "127.12.138.253" port: 42467 } attrs { promote: true } } peers { permanent_uuid: "3b8a3f24429b46a29f1697d97b2da6d1" member_type: NON_VOTER last_known_addr { host: "127.12.138.252" port: 42917 } attrs { promote: true } } }
I20260811 02:52:22.144307 20791 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0317e693c61849e9bf42d55fbf6ef321 [sys.catalog]: This master's current role is: LEADER
I20260811 02:52:22.144799 20851 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 269d7f529b994b32bc6d6d1b512371d4 [term 1 LEARNER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER 3b8a3f24429b46a29f1697d97b2da6d1 (127.12.138.252) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "0317e693c61849e9bf42d55fbf6ef321" member_type: VOTER last_known_addr { host: "127.12.138.254" port: 43457 } } peers { permanent_uuid: "269d7f529b994b32bc6d6d1b512371d4" member_type: NON_VOTER last_known_addr { host: "127.12.138.253" port: 42467 } attrs { promote: true } } peers { permanent_uuid: "3b8a3f24429b46a29f1697d97b2da6d1" member_type: NON_VOTER last_known_addr { host: "127.12.138.252" port: 42917 } attrs { promote: true } } }
W20260811 02:52:22.145342 20117 consensus_peers.cc:487] T 00000000000000000000000000000000 P 0317e693c61849e9bf42d55fbf6ef321 -> Peer 3b8a3f24429b46a29f1697d97b2da6d1 (127.12.138.252:42917): Couldn't send request to peer 3b8a3f24429b46a29f1697d97b2da6d1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260811 02:52:22.146790 20805 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 3b8a3f24429b46a29f1697d97b2da6d1 [term 1 NON_PARTICIPANT]: Raft consensus shutting down.
I20260811 02:52:22.147073 20805 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 3b8a3f24429b46a29f1697d97b2da6d1 [term 1 NON_PARTICIPANT]: Raft consensus is shut down!
I20260811 02:52:22.147212 20805 tablet_replica.cc:330] T 00000000000000000000000000000000 P 3b8a3f24429b46a29f1697d97b2da6d1: stopping tablet replica
I20260811 02:52:22.147330 20883 sys_catalog.cc:455] T 00000000000000000000000000000000 P 269d7f529b994b32bc6d6d1b512371d4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0317e693c61849e9bf42d55fbf6ef321. Latest consensus state: current_term: 1 leader_uuid: "0317e693c61849e9bf42d55fbf6ef321" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "0317e693c61849e9bf42d55fbf6ef321" member_type: VOTER last_known_addr { host: "127.12.138.254" port: 43457 } } peers { permanent_uuid: "269d7f529b994b32bc6d6d1b512371d4" member_type: NON_VOTER last_known_addr { host: "127.12.138.253" port: 42467 } attrs { promote: true } } peers { permanent_uuid: "3b8a3f24429b46a29f1697d97b2da6d1" member_type: NON_VOTER last_known_addr { host: "127.12.138.252" port: 42917 } attrs { promote: true } } }
I20260811 02:52:22.147456 20883 sys_catalog.cc:458] T 00000000000000000000000000000000 P 269d7f529b994b32bc6d6d1b512371d4 [sys.catalog]: This master's current role is: LEARNER
I20260811 02:52:22.147550 20883 sys_catalog.cc:455] T 00000000000000000000000000000000 P 269d7f529b994b32bc6d6d1b512371d4 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "0317e693c61849e9bf42d55fbf6ef321" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "0317e693c61849e9bf42d55fbf6ef321" member_type: VOTER last_known_addr { host: "127.12.138.254" port: 43457 } } peers { permanent_uuid: "269d7f529b994b32bc6d6d1b512371d4" member_type: NON_VOTER last_known_addr { host: "127.12.138.253" port: 42467 } attrs { promote: true } } peers { permanent_uuid: "3b8a3f24429b46a29f1697d97b2da6d1" member_type: NON_VOTER last_known_addr { host: "127.12.138.252" port: 42917 } attrs { promote: true } } }
I20260811 02:52:22.147629 20883 sys_catalog.cc:458] T 00000000000000000000000000000000 P 269d7f529b994b32bc6d6d1b512371d4 [sys.catalog]: This master's current role is: LEARNER
I20260811 02:52:22.150588 20791 raft_consensus.cc:1050] T 00000000000000000000000000000000 P 0317e693c61849e9bf42d55fbf6ef321: attempting to promote NON_VOTER 269d7f529b994b32bc6d6d1b512371d4 to VOTER
I20260811 02:52:22.151150 20791 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0317e693c61849e9bf42d55fbf6ef321 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "0317e693c61849e9bf42d55fbf6ef321" member_type: VOTER last_known_addr { host: "127.12.138.254" port: 43457 } } peers { permanent_uuid: "269d7f529b994b32bc6d6d1b512371d4" member_type: VOTER last_known_addr { host: "127.12.138.253" port: 42467 } attrs { promote: false } } peers { permanent_uuid: "3b8a3f24429b46a29f1697d97b2da6d1" member_type: NON_VOTER last_known_addr { host: "127.12.138.252" port: 42917 } attrs { promote: true } }
I20260811 02:52:22.152529 20788 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:52:22.152246 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.138.252:42917 (local address 127.0.0.1:33468)
0811 02:52:22.152364 (+ 118us) negotiation.cc:107] Waiting for socket to connect
0811 02:52:22.152443 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.138.252:42917: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
I20260811 02:52:22.152539 20851 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 269d7f529b994b32bc6d6d1b512371d4 [term 1 LEARNER]: Refusing update from remote peer 0317e693c61849e9bf42d55fbf6ef321: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20260811 02:52:22.153291 20791 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 0317e693c61849e9bf42d55fbf6ef321 [LEADER]: Connected to new peer: Peer: permanent_uuid: "269d7f529b994b32bc6d6d1b512371d4" member_type: VOTER last_known_addr { host: "127.12.138.253" port: 42467 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260811 02:52:22.153371 20788 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:52:22.153047 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.138.252:42917 (local address 127.0.0.1:33482)
0811 02:52:22.153217 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0811 02:52:22.153272 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.138.252:42917: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":78}
W20260811 02:52:22.153687 20117 consensus_peers.cc:487] T 00000000000000000000000000000000 P 0317e693c61849e9bf42d55fbf6ef321 -> Peer 3b8a3f24429b46a29f1697d97b2da6d1 (127.12.138.252:42917): Couldn't send request to peer 3b8a3f24429b46a29f1697d97b2da6d1. Status: Network error: Client connection negotiation failed: client connection to 127.12.138.252:42917: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260811 02:52:22.155208 20791 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 0317e693c61849e9bf42d55fbf6ef321 [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, 269d7f529b994b32bc6d6d1b512371d4 (127.12.138.253) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "0317e693c61849e9bf42d55fbf6ef321" member_type: VOTER last_known_addr { host: "127.12.138.254" port: 43457 } } peers { permanent_uuid: "269d7f529b994b32bc6d6d1b512371d4" member_type: VOTER last_known_addr { host: "127.12.138.253" port: 42467 } attrs { promote: false } } peers { permanent_uuid: "3b8a3f24429b46a29f1697d97b2da6d1" member_type: NON_VOTER last_known_addr { host: "127.12.138.252" port: 42917 } attrs { promote: true } } }
I20260811 02:52:22.155588 20849 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 269d7f529b994b32bc6d6d1b512371d4 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, 269d7f529b994b32bc6d6d1b512371d4 (127.12.138.253) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "0317e693c61849e9bf42d55fbf6ef321" member_type: VOTER last_known_addr { host: "127.12.138.254" port: 43457 } } peers { permanent_uuid: "269d7f529b994b32bc6d6d1b512371d4" member_type: VOTER last_known_addr { host: "127.12.138.253" port: 42467 } attrs { promote: false } } peers { permanent_uuid: "3b8a3f24429b46a29f1697d97b2da6d1" member_type: NON_VOTER last_known_addr { host: "127.12.138.252" port: 42917 } attrs { promote: true } } }
I20260811 02:52:22.156761 20791 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0317e693c61849e9bf42d55fbf6ef321 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "0317e693c61849e9bf42d55fbf6ef321" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "0317e693c61849e9bf42d55fbf6ef321" member_type: VOTER last_known_addr { host: "127.12.138.254" port: 43457 } } peers { permanent_uuid: "269d7f529b994b32bc6d6d1b512371d4" member_type: VOTER last_known_addr { host: "127.12.138.253" port: 42467 } attrs { promote: false } } peers { permanent_uuid: "3b8a3f24429b46a29f1697d97b2da6d1" member_type: NON_VOTER last_known_addr { host: "127.12.138.252" port: 42917 } attrs { promote: true } } }
I20260811 02:52:22.156891 20791 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0317e693c61849e9bf42d55fbf6ef321 [sys.catalog]: This master's current role is: LEADER
I20260811 02:52:22.157038 20791 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0317e693c61849e9bf42d55fbf6ef321 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "0317e693c61849e9bf42d55fbf6ef321" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "0317e693c61849e9bf42d55fbf6ef321" member_type: VOTER last_known_addr { host: "127.12.138.254" port: 43457 } } peers { permanent_uuid: "269d7f529b994b32bc6d6d1b512371d4" member_type: VOTER last_known_addr { host: "127.12.138.253" port: 42467 } attrs { promote: false } } peers { permanent_uuid: "3b8a3f24429b46a29f1697d97b2da6d1" member_type: NON_VOTER last_known_addr { host: "127.12.138.252" port: 42917 } attrs { promote: true } } }
I20260811 02:52:22.157064 21015 sys_catalog.cc:455] T 00000000000000000000000000000000 P 269d7f529b994b32bc6d6d1b512371d4 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "0317e693c61849e9bf42d55fbf6ef321" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "0317e693c61849e9bf42d55fbf6ef321" member_type: VOTER last_known_addr { host: "127.12.138.254" port: 43457 } } peers { permanent_uuid: "269d7f529b994b32bc6d6d1b512371d4" member_type: VOTER last_known_addr { host: "127.12.138.253" port: 42467 } attrs { promote: false } } peers { permanent_uuid: "3b8a3f24429b46a29f1697d97b2da6d1" member_type: NON_VOTER last_known_addr { host: "127.12.138.252" port: 42917 } attrs { promote: true } } }
I20260811 02:52:22.157220 21015 sys_catalog.cc:458] T 00000000000000000000000000000000 P 269d7f529b994b32bc6d6d1b512371d4 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 02:52:22.157339 20805 master.cc:552] Master@127.12.138.252:42917 shutdown complete.
I20260811 02:52:22.157392 20805 master_runner.cc:305] Clearing existing system tablet
I20260811 02:52:22.157658 20805 fs_manager.cc:359] Metadata directory not provided
I20260811 02:52:22.157742 20805 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 02:52:22.157125 20791 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0317e693c61849e9bf42d55fbf6ef321 [sys.catalog]: This master's current role is: LEADER
I20260811 02:52:22.158882 20805 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260811 02:52:22.159861 21017 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 02:52:22.160095 20805 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786416707705494-12843-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 02:52:22.160249 20805 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 02:52:22.160288 20805 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786416707705494-12843-0/minicluster-data/master-2/data,/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786416707705494-12843-0/minicluster-data/master-2/wal
uuid: "3b8a3f24429b46a29f1697d97b2da6d1"
format_stamp: "Formatted at 2026-08-11 02:52:21 on dist-test-slave-9gcw"
server_key: "169af108765a2c182d59563c54fa41ab"
server_key_iv: "f75faa6503ffd7bd99a64f544c79bcf2"
server_key_version: "encryptionkey@0"
I20260811 02:52:22.161840 20805 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P 3b8a3f24429b46a29f1697d97b2da6d1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 02:52:22.163846 20805 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P 3b8a3f24429b46a29f1697d97b2da6d1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260811 02:52:22.163950 20805 log.cc:1196] T 00000000000000000000000000000000 P 3b8a3f24429b46a29f1697d97b2da6d1: Deleting WAL directory at /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786416707705494-12843-0/minicluster-data/master-2/wal/wals/00000000000000000000000000000000
I20260811 02:52:22.164327 20805 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P 3b8a3f24429b46a29f1697d97b2da6d1: Deleting consensus metadata
I20260811 02:52:22.164515 20805 master_runner.cc:315] Copying system tablet from 127.12.138.254:43457
I20260811 02:52:22.165706 20805 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P 3b8a3f24429b46a29f1697d97b2da6d1: tablet copy: Beginning tablet copy session from remote peer at address 127.12.138.254:43457
I20260811 02:52:22.177022 20182 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:52:22.166328 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54594 (local address 127.12.138.254:43457)
0811 02:52:22.166471 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0811 02:52:22.166475 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 02:52:22.169531 (+ 3056us) server_negotiation.cc:407] Connection header received
0811 02:52:22.169781 (+ 250us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:52:22.169784 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:52:22.169831 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:52:22.169935 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:52:22.170447 (+ 512us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:52:22.170990 (+ 543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:52:22.171941 (+ 951us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:52:22.172171 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:52:22.172286 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:52:22.173548 (+ 1262us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:52:22.173553 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:52:22.173559 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:52:22.173586 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:52:22.175222 (+ 1636us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:52:22.175765 (+ 543us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:52:22.175769 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:52:22.175771 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:52:22.175858 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:52:22.176139 (+ 281us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:52:22.176145 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:52:22.176150 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:52:22.176350 (+ 200us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:52:22.176461 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0811 02:52:22.176747 (+ 286us) server_negotiation.cc:299] Negotiation successful
0811 02:52:22.176863 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260811 02:52:22.177033 21023 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:52:22.166255 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.138.254:43457 (local address 127.0.0.1:54594)
0811 02:52:22.169417 (+ 3162us) negotiation.cc:107] Waiting for socket to connect
0811 02:52:22.169432 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 02:52:22.169584 (+ 152us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:52:22.169958 (+ 374us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:52:22.169961 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:52:22.170216 (+ 255us) client_negotiation.cc:189] Negotiated authn=SASL
0811 02:52:22.170319 (+ 103us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:52:22.170325 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:52:22.171110 (+ 785us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:52:22.171116 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:52:22.171816 (+ 700us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:52:22.171825 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:52:22.172293 (+ 468us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:52:22.172297 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:52:22.172394 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:52:22.172963 (+ 569us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 02:52:22.172984 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0811 02:52:22.173403 (+ 419us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 02:52:22.175394 (+ 1991us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:52:22.175436 (+ 42us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:52:22.175438 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:52:22.175654 (+ 216us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:52:22.175952 (+ 298us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:52:22.175956 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:52:22.175958 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:52:22.176048 (+ 90us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:52:22.176450 (+ 402us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 02:52:22.176453 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 02:52:22.176562 (+ 109us) client_negotiation.cc:769] Sending connection context
0811 02:52:22.176683 (+ 121us) client_negotiation.cc:240] Negotiation successful
0811 02:52:22.176838 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3076,"thread_start_us":90,"threads_started":1}
I20260811 02:52:22.178161 20160 tablet_copy_service.cc:140] P 0317e693c61849e9bf42d55fbf6ef321: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer 3b8a3f24429b46a29f1697d97b2da6d1 ({username='oryx', principal='oryx/127.12.138.252@KRBTEST.COM'} at 127.0.0.1:54594)
I20260811 02:52:22.178308 20160 tablet_copy_service.cc:161] P 0317e693c61849e9bf42d55fbf6ef321: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer 3b8a3f24429b46a29f1697d97b2da6d1 at {username='oryx', principal='oryx/127.12.138.252@KRBTEST.COM'} at 127.0.0.1:54594: session id = 3b8a3f24429b46a29f1697d97b2da6d1-00000000000000000000000000000000
I20260811 02:52:22.179327 20160 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P 0317e693c61849e9bf42d55fbf6ef321: Tablet Copy: opened 0 blocks and 1 log segments
I20260811 02:52:22.180699 20805 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260811 02:52:22.183238 20805 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P 3b8a3f24429b46a29f1697d97b2da6d1: tablet copy: Starting download of 0 data blocks...
I20260811 02:52:22.183543 20805 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P 3b8a3f24429b46a29f1697d97b2da6d1: tablet copy: Starting download of 1 WAL segments...
I20260811 02:52:22.185247 20805 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P 3b8a3f24429b46a29f1697d97b2da6d1: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260811 02:52:22.186959 20160 tablet_copy_service.cc:342] P 0317e693c61849e9bf42d55fbf6ef321: Request end of tablet copy session 3b8a3f24429b46a29f1697d97b2da6d1-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.12.138.252@KRBTEST.COM'} at 127.0.0.1:54594
I20260811 02:52:22.187074 20160 tablet_copy_service.cc:433] P 0317e693c61849e9bf42d55fbf6ef321: ending tablet copy session 3b8a3f24429b46a29f1697d97b2da6d1-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer 3b8a3f24429b46a29f1697d97b2da6d1
I20260811 02:52:22.194378 20805 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 02:52:22.196230 21027 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:52:22.196252 21024 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:52:22.196564 21025 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:52:22.196775 20805 server_base.cc:1034] running on GCE node
I20260811 02:52:22.197010 20805 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 02:52:22.197431 20805 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 02:52:22.198575 20805 hybrid_clock.cc:648] HybridClock initialized: now 1786416742198559 us; error 39 us; skew 500 ppm
Aug 11 02:52:22 dist-test-slave-9gcw krb5kdc[20077](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416742, etypes {rep=17 tkt=17 ses=17}, oryx/127.12.138.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 02:52:22.200281 20805 init.cc:377] Logged in from keytab as oryx/127.12.138.252@KRBTEST.COM (short username oryx)
I20260811 02:52:22.200888 20805 webserver.cc:460] Webserver started at http://127.12.138.252:46305/ using document root <none> and password file <none>
I20260811 02:52:22.201126 20805 fs_manager.cc:359] Metadata directory not provided
I20260811 02:52:22.201234 20805 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 02:52:22.202662 20805 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260811 02:52:22.203737 21036 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 02:52:22.203992 20805 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 02:52:22.204097 20805 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786416707705494-12843-0/minicluster-data/master-2/data,/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786416707705494-12843-0/minicluster-data/master-2/wal
uuid: "3b8a3f24429b46a29f1697d97b2da6d1"
format_stamp: "Formatted at 2026-08-11 02:52:21 on dist-test-slave-9gcw"
server_key: "169af108765a2c182d59563c54fa41ab"
server_key_iv: "f75faa6503ffd7bd99a64f544c79bcf2"
server_key_version: "encryptionkey@0"
I20260811 02:52:22.204217 20805 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786416707705494-12843-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786416707705494-12843-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786416707705494-12843-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 02:52:22.208818 20805 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 02:52:22.211140 20805 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 02:52:22.217746 20805 rpc_server.cc:307] RPC server started. Bound to: 127.12.138.252:42917
I20260811 02:52:22.218631 20805 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786416707705494-12843-0/minicluster-data/master-2/data/info.pb
I20260811 02:52:22.219135 21100 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.138.252:42917 every 8 connection(s)
I20260811 02:52:22.220402 21101 sys_catalog.cc:263] Verifying existing consensus state
I20260811 02:52:22.221427 21101 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3b8a3f24429b46a29f1697d97b2da6d1: Bootstrap starting.
I20260811 02:52:22.228636 21101 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3b8a3f24429b46a29f1697d97b2da6d1: Bootstrap replayed 1/1 log segments. Stats: ops{read=7 overwritten=0 applied=7 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 02:52:22.228981 21101 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3b8a3f24429b46a29f1697d97b2da6d1: Bootstrap complete.
I20260811 02:52:22.229619 21101 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3b8a3f24429b46a29f1697d97b2da6d1 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "0317e693c61849e9bf42d55fbf6ef321" member_type: VOTER last_known_addr { host: "127.12.138.254" port: 43457 } } peers { permanent_uuid: "269d7f529b994b32bc6d6d1b512371d4" member_type: VOTER last_known_addr { host: "127.12.138.253" port: 42467 } attrs { promote: false } } peers { permanent_uuid: "3b8a3f24429b46a29f1697d97b2da6d1" member_type: NON_VOTER last_known_addr { host: "127.12.138.252" port: 42917 } attrs { promote: true } }
I20260811 02:52:22.229771 21101 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3b8a3f24429b46a29f1697d97b2da6d1 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 3b8a3f24429b46a29f1697d97b2da6d1, State: Initialized, Role: LEARNER
I20260811 02:52:22.229929 21101 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3b8a3f24429b46a29f1697d97b2da6d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 7, Last appended: 1.7, Last appended by leader: 7, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "0317e693c61849e9bf42d55fbf6ef321" member_type: VOTER last_known_addr { host: "127.12.138.254" port: 43457 } } peers { permanent_uuid: "269d7f529b994b32bc6d6d1b512371d4" member_type: VOTER last_known_addr { host: "127.12.138.253" port: 42467 } attrs { promote: false } } peers { permanent_uuid: "3b8a3f24429b46a29f1697d97b2da6d1" member_type: NON_VOTER last_known_addr { host: "127.12.138.252" port: 42917 } attrs { promote: true } }
I20260811 02:52:22.230317 21104 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3b8a3f24429b46a29f1697d97b2da6d1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "0317e693c61849e9bf42d55fbf6ef321" member_type: VOTER last_known_addr { host: "127.12.138.254" port: 43457 } } peers { permanent_uuid: "269d7f529b994b32bc6d6d1b512371d4" member_type: VOTER last_known_addr { host: "127.12.138.253" port: 42467 } attrs { promote: false } } peers { permanent_uuid: "3b8a3f24429b46a29f1697d97b2da6d1" member_type: NON_VOTER last_known_addr { host: "127.12.138.252" port: 42917 } attrs { promote: true } } }
I20260811 02:52:22.230494 21101 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3b8a3f24429b46a29f1697d97b2da6d1 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 02:52:22.230538 21104 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3b8a3f24429b46a29f1697d97b2da6d1 [sys.catalog]: This master's current role is: LEARNER
I20260811 02:52:22.234689 21116 catalog_manager.cc:1259] Loaded cluster ID: 101bfd3e0cf345c7b289e94b98d49ad6
I20260811 02:52:22.234779 21116 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 3b8a3f24429b46a29f1697d97b2da6d1: loading cluster ID for follower catalog manager: success
I20260811 02:52:22.236991 21116 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 3b8a3f24429b46a29f1697d97b2da6d1: acquiring CA information for follower catalog manager: success
I20260811 02:52:22.238500 21116 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 3b8a3f24429b46a29f1697d97b2da6d1: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260811 02:52:22.258140 20512 heartbeater.cc:503] Master 127.12.138.254:43457 was elected leader, sending a full tablet report...
W20260811 02:52:22.297325 21004 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 02:52:22.297624 21004 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 02:52:22.297674 21004 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 02:52:22.302114 21004 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 02:52:22.302203 21004 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 02:52:22.302304 21004 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.138.193
I20260811 02:52:22.308095 21004 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.138.212:41877
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786416707705494-12843-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786416707705494-12843-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786416707705494-12843-0/krb5kdc/oryx_127.12.138.193.keytab
--principal=oryx/127.12.138.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.138.193:41461
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786416707705494-12843-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.12.138.193
--webserver_port=35107
--tserver_master_addrs=127.12.138.254:43457,127.12.138.253:42467,127.12.138.252:42917
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21004
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.138.193
--log_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786416707705494-12843-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 02:09:49 UTC on ccycloud.quasar-fsnqqc.root.comops.site
build id 4579570
I20260811 02:52:22.309377 21004 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 02:52:22.310616 21004 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 02:52:22.321658 21122 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:52:22.321753 21124 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:52:22.321702 21004 server_base.cc:1034] running on GCE node
W20260811 02:52:22.321722 21121 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:52:22.322260 21004 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 02:52:22.322882 21004 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 02:52:22.324074 21004 hybrid_clock.cc:648] HybridClock initialized: now 1786416742324045 us; error 42 us; skew 500 ppm
Aug 11 02:52:22 dist-test-slave-9gcw krb5kdc[20077](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416742, etypes {rep=17 tkt=17 ses=17}, oryx/127.12.138.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 02:52:22.326998 21004 init.cc:377] Logged in from keytab as oryx/127.12.138.193@KRBTEST.COM (short username oryx)
I20260811 02:52:22.327857 21004 webserver.cc:460] Webserver started at http://127.12.138.193:35107/ using document root <none> and password file <none>
I20260811 02:52:22.328395 21004 fs_manager.cc:359] Metadata directory not provided
I20260811 02:52:22.328470 21004 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 02:52:22.332842 21004 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.000s
I20260811 02:52:22.334956 21131 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 02:52:22.336234 21004 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260811 02:52:22.336375 21004 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786416707705494-12843-0/minicluster-data/ts-0/data,/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786416707705494-12843-0/minicluster-data/ts-0/wal
uuid: "ab096c76c5544f828e9eb7f2bee1977e"
format_stamp: "Formatted at 2026-08-11 02:52:20 on dist-test-slave-9gcw"
server_key: "8f4c91db5f4edf38b243a7cece2d51da"
server_key_iv: "b7761123df6c99c49ea923613fece180"
server_key_version: "encryptionkey@0"
I20260811 02:52:22.336941 21004 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786416707705494-12843-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786416707705494-12843-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786416707705494-12843-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 02:52:22.355374 21004 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 02:52:22.358855 21004 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 02:52:22.359079 21004 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 02:52:22.359709 21004 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 02:52:22.360664 21004 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 02:52:22.360713 21004 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 02:52:22.360754 21004 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 02:52:22.360782 21004 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 02:52:22.372179 21004 rpc_server.cc:307] RPC server started. Bound to: 127.12.138.193:41461
I20260811 02:52:22.372226 21259 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.138.193:41461 every 8 connection(s)
I20260811 02:52:22.373349 21004 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786416707705494-12843-0/minicluster-data/ts-0/data/info.pb
I20260811 02:52:22.374913 12843 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDawEwS/build/debug/bin/kudu as pid 21004
I20260811 02:52:22.375125 12843 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDawEwS/build/debug/bin/kudu with pid 20365
I20260811 02:52:22.384953 12843 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDawEwS/build/debug/bin/kudu
/tmp/dist-test-taskDawEwS/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786416707705494-12843-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786416707705494-12843-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786416707705494-12843-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786416707705494-12843-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.138.194:40543
--local_ip_for_outbound_sockets=127.12.138.194
--tserver_master_addrs=127.12.138.254:43457
--webserver_port=39213
--webserver_interface=127.12.138.194
--keytab_file=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786416707705494-12843-0/krb5kdc/oryx_127.12.138.194.keytab
--principal=oryx/127.12.138.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.138.212:41877
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.12.138.254:43457,127.12.138.253:42467
--tserver_master_addrs=127.12.138.254:43457,127.12.138.253:42467,127.12.138.252:42917 with env {KRB5CCNAME=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786416707705494-12843-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786416707705494-12843-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786416707705494-12843-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 11 02:52:22 dist-test-slave-9gcw krb5kdc[20077](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416742, etypes {rep=17 tkt=17 ses=17}, oryx/127.12.138.193@KRBTEST.COM for oryx/127.12.138.254@KRBTEST.COM
Aug 11 02:52:22 dist-test-slave-9gcw krb5kdc[20077](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416742, etypes {rep=17 tkt=17 ses=17}, oryx/127.12.138.193@KRBTEST.COM for oryx/127.12.138.252@KRBTEST.COM
Aug 11 02:52:22 dist-test-slave-9gcw krb5kdc[20077](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416742, etypes {rep=17 tkt=17 ses=17}, oryx/127.12.138.193@KRBTEST.COM for oryx/127.12.138.253@KRBTEST.COM
I20260811 02:52:22.393893 20182 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:52:22.379050 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.138.193:44823 (local address 127.12.138.254:43457)
0811 02:52:22.379223 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0811 02:52:22.379227 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 02:52:22.379245 (+ 18us) server_negotiation.cc:407] Connection header received
0811 02:52:22.379755 (+ 510us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:52:22.379759 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:52:22.379821 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:52:22.379881 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:52:22.382248 (+ 2367us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:52:22.383126 (+ 878us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:52:22.385229 (+ 2103us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:52:22.385572 (+ 343us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:52:22.385653 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:52:22.389006 (+ 3353us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:52:22.389010 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:52:22.389014 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:52:22.389050 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:52:22.391116 (+ 2066us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:52:22.391803 (+ 687us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:52:22.391810 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:52:22.391813 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:52:22.391927 (+ 114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:52:22.392406 (+ 479us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:52:22.392410 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:52:22.392412 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:52:22.392669 (+ 257us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:52:22.392748 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0811 02:52:22.393561 (+ 813us) server_negotiation.cc:299] Negotiation successful
0811 02:52:22.393713 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260811 02:52:22.394327 21266 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:52:22.376805 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.138.193:41561 (local address 127.12.138.252:42917)
0811 02:52:22.377081 (+ 276us) server_negotiation.cc:206] Beginning negotiation
0811 02:52:22.377085 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 02:52:22.378621 (+ 1536us) server_negotiation.cc:407] Connection header received
0811 02:52:22.379756 (+ 1135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:52:22.379760 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:52:22.379821 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:52:22.379890 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:52:22.382248 (+ 2358us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:52:22.383373 (+ 1125us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:52:22.385943 (+ 2570us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:52:22.386375 (+ 432us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:52:22.386472 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:52:22.389726 (+ 3254us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:52:22.389731 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:52:22.389735 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:52:22.389767 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:52:22.391981 (+ 2214us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:52:22.392535 (+ 554us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:52:22.392542 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:52:22.392546 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:52:22.392649 (+ 103us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:52:22.393031 (+ 382us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:52:22.393059 (+ 28us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:52:22.393062 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:52:22.393371 (+ 309us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:52:22.393476 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0811 02:52:22.393911 (+ 435us) server_negotiation.cc:299] Negotiation successful
0811 02:52:22.394073 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":198,"thread_start_us":81,"threads_started":1}
I20260811 02:52:22.395093 20977 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:52:22.377004 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.138.193:49463 (local address 127.12.138.253:42467)
0811 02:52:22.377183 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0811 02:52:22.377190 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 02:52:22.379510 (+ 2320us) server_negotiation.cc:407] Connection header received
0811 02:52:22.379756 (+ 246us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:52:22.379762 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:52:22.379827 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:52:22.379885 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:52:22.382240 (+ 2355us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:52:22.383162 (+ 922us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:52:22.386867 (+ 3705us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:52:22.387219 (+ 352us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:52:22.387308 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:52:22.390301 (+ 2993us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:52:22.390306 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:52:22.390311 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:52:22.390349 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:52:22.392822 (+ 2473us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:52:22.393321 (+ 499us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:52:22.393326 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:52:22.393328 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:52:22.393413 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:52:22.393722 (+ 309us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:52:22.393726 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:52:22.393727 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:52:22.393945 (+ 218us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:52:22.394029 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0811 02:52:22.394825 (+ 796us) server_negotiation.cc:299] Negotiation successful
0811 02:52:22.394946 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260811 02:52:22.395880 21265 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:52:22.375701 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.138.253:42467 (local address 127.12.138.193:49463)
0811 02:52:22.379405 (+ 3704us) negotiation.cc:107] Waiting for socket to connect
0811 02:52:22.379422 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 02:52:22.379585 (+ 163us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:52:22.380390 (+ 805us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:52:22.380396 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:52:22.381108 (+ 712us) client_negotiation.cc:189] Negotiated authn=SASL
0811 02:52:22.382041 (+ 933us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:52:22.382066 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:52:22.383334 (+ 1268us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:52:22.383340 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:52:22.385767 (+ 2427us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:52:22.385782 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:52:22.387473 (+ 1691us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:52:22.387477 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:52:22.387585 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:52:22.387997 (+ 412us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 02:52:22.388011 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0811 02:52:22.390142 (+ 2131us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 02:52:22.392932 (+ 2790us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:52:22.392935 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:52:22.392936 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:52:22.393201 (+ 265us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:52:22.393526 (+ 325us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:52:22.393529 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:52:22.393531 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:52:22.393611 (+ 80us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:52:22.394304 (+ 693us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 02:52:22.394307 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 02:52:22.394611 (+ 304us) client_negotiation.cc:769] Sending connection context
0811 02:52:22.395482 (+ 871us) client_negotiation.cc:240] Negotiation successful
0811 02:52:22.395621 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3586,"mutex_wait_us":3324,"spinlock_wait_cycles":16256,"thread_start_us":128,"threads_started":1}
I20260811 02:52:22.396193 21267 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:52:22.375701 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.138.254:43457 (local address 127.12.138.193:44823)
0811 02:52:22.378371 (+ 2670us) negotiation.cc:107] Waiting for socket to connect
0811 02:52:22.378434 (+ 63us) client_negotiation.cc:174] Beginning negotiation
0811 02:52:22.379490 (+ 1056us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:52:22.380328 (+ 838us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:52:22.380338 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:52:22.381109 (+ 771us) client_negotiation.cc:189] Negotiated authn=SASL
0811 02:52:22.382039 (+ 930us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:52:22.382061 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:52:22.383311 (+ 1250us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:52:22.383314 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:52:22.385008 (+ 1694us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:52:22.385026 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:52:22.385897 (+ 871us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:52:22.385900 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:52:22.386088 (+ 188us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:52:22.386802 (+ 714us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 02:52:22.386824 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0811 02:52:22.388831 (+ 2007us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 02:52:22.391262 (+ 2431us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:52:22.391271 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:52:22.391304 (+ 33us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:52:22.391672 (+ 368us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:52:22.392048 (+ 376us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:52:22.392052 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:52:22.392055 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:52:22.392262 (+ 207us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:52:22.392843 (+ 581us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 02:52:22.392852 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 02:52:22.393239 (+ 387us) client_negotiation.cc:769] Sending connection context
0811 02:52:22.394159 (+ 920us) client_negotiation.cc:240] Negotiation successful
0811 02:52:22.394672 (+ 513us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2379,"mutex_wait_us":282,"spinlock_wait_cycles":36096,"thread_start_us":843,"threads_started":1}
I20260811 02:52:22.395884 21268 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:52:22.375701 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.138.252:42917 (local address 127.12.138.193:41561)
0811 02:52:22.378507 (+ 2806us) negotiation.cc:107] Waiting for socket to connect
0811 02:52:22.378521 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 02:52:22.379490 (+ 969us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:52:22.380328 (+ 838us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:52:22.380338 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:52:22.381119 (+ 781us) client_negotiation.cc:189] Negotiated authn=SASL
0811 02:52:22.382039 (+ 920us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:52:22.382061 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:52:22.383510 (+ 1449us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:52:22.383513 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:52:22.385012 (+ 1499us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:52:22.385025 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:52:22.386701 (+ 1676us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:52:22.386707 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:52:22.386845 (+ 138us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:52:22.387349 (+ 504us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 02:52:22.387363 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0811 02:52:22.389544 (+ 2181us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 02:52:22.392101 (+ 2557us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:52:22.392107 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:52:22.392112 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:52:22.392398 (+ 286us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:52:22.392795 (+ 397us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:52:22.392800 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:52:22.392805 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:52:22.392908 (+ 103us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:52:22.393547 (+ 639us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 02:52:22.393553 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 02:52:22.393665 (+ 112us) client_negotiation.cc:769] Sending connection context
0811 02:52:22.394493 (+ 828us) client_negotiation.cc:240] Negotiation successful
0811 02:52:22.394763 (+ 270us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2707,"mutex_wait_us":2643,"thread_start_us":114,"threads_started":1}
I20260811 02:52:22.397529 21262 heartbeater.cc:347] Connected to a master server at 127.12.138.253:42467
I20260811 02:52:22.397569 21260 heartbeater.cc:347] Connected to a master server at 127.12.138.252:42917
I20260811 02:52:22.397571 21261 heartbeater.cc:347] Connected to a master server at 127.12.138.254:43457
I20260811 02:52:22.397786 21262 heartbeater.cc:464] Registering TS with master...
I20260811 02:52:22.397785 21260 heartbeater.cc:464] Registering TS with master...
I20260811 02:52:22.397785 21261 heartbeater.cc:464] Registering TS with master...
I20260811 02:52:22.398417 21260 heartbeater.cc:511] Master 127.12.138.252:42917 requested a full tablet report, sending...
I20260811 02:52:22.398432 21262 heartbeater.cc:511] Master 127.12.138.253:42467 requested a full tablet report, sending...
I20260811 02:52:22.398417 21261 heartbeater.cc:511] Master 127.12.138.254:43457 requested a full tablet report, sending...
I20260811 02:52:22.399729 20136 ts_manager.cc:194] Re-registered known tserver with Master: ab096c76c5544f828e9eb7f2bee1977e (127.12.138.193:41461)
I20260811 02:52:22.399986 21056 ts_manager.cc:194] Registered new tserver with Master: ab096c76c5544f828e9eb7f2bee1977e (127.12.138.193:41461)
I20260811 02:52:22.400144 20830 ts_manager.cc:194] Registered new tserver with Master: ab096c76c5544f828e9eb7f2bee1977e (127.12.138.193:41461)
I20260811 02:52:22.400506 20136 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.12.138.193@KRBTEST.COM'} at 127.12.138.193:44823
I20260811 02:52:22.484408 20674 heartbeater.cc:503] Master 127.12.138.254:43457 was elected leader, sending a full tablet report...
W20260811 02:52:22.512064 21269 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 02:52:22.512400 21269 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 02:52:22.512475 21269 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 02:52:22.516633 21269 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 02:52:22.516721 21269 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 02:52:22.516811 21269 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.138.194
I20260811 02:52:22.522296 21269 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.138.212:41877
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786416707705494-12843-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786416707705494-12843-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786416707705494-12843-0/krb5kdc/oryx_127.12.138.194.keytab
--principal=oryx/127.12.138.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.138.194:40543
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786416707705494-12843-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.12.138.194
--webserver_port=39213
--tserver_master_addrs=127.12.138.254:43457,127.12.138.253:42467,127.12.138.252:42917
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21269
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.138.194
--log_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786416707705494-12843-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 02:09:49 UTC on ccycloud.quasar-fsnqqc.root.comops.site
build id 4579570
I20260811 02:52:22.523375 21269 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 02:52:22.524446 21269 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 02:52:22.535176 21277 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:52:22.535264 21269 server_base.cc:1034] running on GCE node
W20260811 02:52:22.535219 21274 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:52:22.535219 21275 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:52:22.535945 21269 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 02:52:22.536579 21269 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 02:52:22.537741 21269 hybrid_clock.cc:648] HybridClock initialized: now 1786416742537707 us; error 46 us; skew 500 ppm
Aug 11 02:52:22 dist-test-slave-9gcw krb5kdc[20077](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416742, etypes {rep=17 tkt=17 ses=17}, oryx/127.12.138.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 02:52:22.540680 21269 init.cc:377] Logged in from keytab as oryx/127.12.138.194@KRBTEST.COM (short username oryx)
I20260811 02:52:22.541831 21269 webserver.cc:460] Webserver started at http://127.12.138.194:39213/ using document root <none> and password file <none>
I20260811 02:52:22.542512 21269 fs_manager.cc:359] Metadata directory not provided
I20260811 02:52:22.542613 21269 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 02:52:22.546958 21269 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260811 02:52:22.549000 21285 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 02:52:22.550302 21269 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260811 02:52:22.550448 21269 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786416707705494-12843-0/minicluster-data/ts-1/data,/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786416707705494-12843-0/minicluster-data/ts-1/wal
uuid: "c51f9680930c4419a91d0d0f521006dd"
format_stamp: "Formatted at 2026-08-11 02:52:21 on dist-test-slave-9gcw"
server_key: "4aa288bff2395caa74179f6d82f5b9b3"
server_key_iv: "85708040c4669db6eba453f80cc6f18c"
server_key_version: "encryptionkey@0"
I20260811 02:52:22.551013 21269 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786416707705494-12843-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786416707705494-12843-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786416707705494-12843-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 02:52:22.568620 21269 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 02:52:22.572266 21269 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 02:52:22.572502 21269 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 02:52:22.573135 21269 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 02:52:22.574093 21269 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 02:52:22.574167 21269 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 02:52:22.574240 21269 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 02:52:22.574292 21269 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 02:52:22.585042 21269 rpc_server.cc:307] RPC server started. Bound to: 127.12.138.194:40543
I20260811 02:52:22.585104 21417 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.138.194:40543 every 8 connection(s)
I20260811 02:52:22.586179 21269 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786416707705494-12843-0/minicluster-data/ts-1/data/info.pb
I20260811 02:52:22.594638 12843 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDawEwS/build/debug/bin/kudu as pid 21269
I20260811 02:52:22.594833 12843 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDawEwS/build/debug/bin/kudu with pid 20517
Aug 11 02:52:22 dist-test-slave-9gcw krb5kdc[20077](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416742, etypes {rep=17 tkt=17 ses=17}, oryx/127.12.138.194@KRBTEST.COM for oryx/127.12.138.252@KRBTEST.COM
Aug 11 02:52:22 dist-test-slave-9gcw krb5kdc[20077](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416742, etypes {rep=17 tkt=17 ses=17}, oryx/127.12.138.194@KRBTEST.COM for oryx/127.12.138.254@KRBTEST.COM
I20260811 02:52:22.604079 21266 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:52:22.588977 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.138.194:45771 (local address 127.12.138.252:42917)
0811 02:52:22.589105 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0811 02:52:22.589110 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 02:52:22.589963 (+ 853us) server_negotiation.cc:407] Connection header received
0811 02:52:22.590925 (+ 962us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:52:22.590929 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:52:22.590984 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:52:22.591040 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:52:22.592440 (+ 1400us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:52:22.593279 (+ 839us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:52:22.594632 (+ 1353us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:52:22.595056 (+ 424us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:52:22.595176 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:52:22.599374 (+ 4198us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:52:22.599378 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:52:22.599381 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:52:22.599415 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:52:22.601729 (+ 2314us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:52:22.602251 (+ 522us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:52:22.602255 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:52:22.602256 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:52:22.602356 (+ 100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:52:22.602807 (+ 451us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:52:22.602810 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:52:22.602812 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:52:22.603006 (+ 194us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:52:22.603105 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0811 02:52:22.603726 (+ 621us) server_negotiation.cc:299] Negotiation successful
0811 02:52:22.603878 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260811 02:52:22.604909 21422 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:52:22.588843 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.138.252:42917 (local address 127.12.138.194:45771)
0811 02:52:22.589864 (+ 1021us) negotiation.cc:107] Waiting for socket to connect
0811 02:52:22.589876 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 02:52:22.590685 (+ 809us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:52:22.591247 (+ 562us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:52:22.591255 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:52:22.591770 (+ 515us) client_negotiation.cc:189] Negotiated authn=SASL
0811 02:52:22.592267 (+ 497us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:52:22.592284 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:52:22.593391 (+ 1107us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:52:22.593394 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:52:22.594509 (+ 1115us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:52:22.594515 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:52:22.595369 (+ 854us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:52:22.595376 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:52:22.595674 (+ 298us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:52:22.596547 (+ 873us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 02:52:22.596575 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0811 02:52:22.599154 (+ 2579us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 02:52:22.601848 (+ 2694us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:52:22.601855 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:52:22.601871 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:52:22.602140 (+ 269us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:52:22.602473 (+ 333us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:52:22.602475 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:52:22.602477 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:52:22.602686 (+ 209us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:52:22.603188 (+ 502us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 02:52:22.603196 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 02:52:22.603486 (+ 290us) client_negotiation.cc:769] Sending connection context
0811 02:52:22.603685 (+ 199us) client_negotiation.cc:240] Negotiation successful
0811 02:52:22.604026 (+ 341us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":971,"mutex_wait_us":186,"spinlock_wait_cycles":16896,"thread_start_us":120,"threads_started":1}
I20260811 02:52:22.605582 20182 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:52:22.588168 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.138.194:51681 (local address 127.12.138.254:43457)
0811 02:52:22.588304 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0811 02:52:22.588308 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 02:52:22.589695 (+ 1387us) server_negotiation.cc:407] Connection header received
0811 02:52:22.590906 (+ 1211us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:52:22.590910 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:52:22.590974 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:52:22.591056 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:52:22.592427 (+ 1371us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:52:22.592978 (+ 551us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:52:22.594657 (+ 1679us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:52:22.595019 (+ 362us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:52:22.595132 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:52:22.600094 (+ 4962us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:52:22.600098 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:52:22.600101 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:52:22.600138 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:52:22.602327 (+ 2189us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:52:22.602833 (+ 506us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:52:22.602840 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:52:22.602845 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:52:22.602958 (+ 113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:52:22.603288 (+ 330us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:52:22.603293 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:52:22.603295 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:52:22.603602 (+ 307us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:52:22.603721 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0811 02:52:22.604810 (+ 1089us) server_negotiation.cc:299] Negotiation successful
0811 02:52:22.604974 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260811 02:52:22.606187 21418 heartbeater.cc:347] Connected to a master server at 127.12.138.252:42917
I20260811 02:52:22.606515 21418 heartbeater.cc:464] Registering TS with master...
I20260811 02:52:22.607319 21418 heartbeater.cc:511] Master 127.12.138.252:42917 requested a full tablet report, sending...
I20260811 02:52:22.608474 21425 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:52:22.588619 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.138.254:43457 (local address 127.12.138.194:51681)
0811 02:52:22.589535 (+ 916us) negotiation.cc:107] Waiting for socket to connect
0811 02:52:22.589588 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0811 02:52:22.590697 (+ 1109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:52:22.591247 (+ 550us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:52:22.591254 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:52:22.591682 (+ 428us) client_negotiation.cc:189] Negotiated authn=SASL
0811 02:52:22.592266 (+ 584us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:52:22.592287 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:52:22.593137 (+ 850us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:52:22.593181 (+ 44us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:52:22.594502 (+ 1321us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:52:22.594514 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:52:22.597643 (+ 3129us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:52:22.597649 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:52:22.597766 (+ 117us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:52:22.598305 (+ 539us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 02:52:22.598317 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0811 02:52:22.599884 (+ 1567us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 02:52:22.602449 (+ 2565us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:52:22.602454 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:52:22.602456 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:52:22.602692 (+ 236us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:52:22.603078 (+ 386us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:52:22.603081 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:52:22.603083 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:52:22.603174 (+ 91us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:52:22.603782 (+ 608us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 02:52:22.603788 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 02:52:22.603993 (+ 205us) client_negotiation.cc:769] Sending connection context
0811 02:52:22.606045 (+ 2052us) client_negotiation.cc:240] Negotiation successful
0811 02:52:22.608152 (+ 2107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":718,"mutex_wait_us":50,"spinlock_wait_cycles":69376,"thread_start_us":241,"threads_started":1}
I20260811 02:52:22.608764 21056 ts_manager.cc:194] Registered new tserver with Master: c51f9680930c4419a91d0d0f521006dd (127.12.138.194:40543)
I20260811 02:52:22.615059 21419 heartbeater.cc:347] Connected to a master server at 127.12.138.254:43457
I20260811 02:52:22.615212 21419 heartbeater.cc:464] Registering TS with master...
I20260811 02:52:22.615479 21419 heartbeater.cc:511] Master 127.12.138.254:43457 requested a full tablet report, sending...
I20260811 02:52:22.616376 20136 ts_manager.cc:194] Re-registered known tserver with Master: c51f9680930c4419a91d0d0f521006dd (127.12.138.194:40543)
I20260811 02:52:22.617228 20136 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.12.138.194@KRBTEST.COM'} at 127.12.138.194:51681
Aug 11 02:52:22 dist-test-slave-9gcw krb5kdc[20077](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416742, etypes {rep=17 tkt=17 ses=17}, oryx/127.12.138.194@KRBTEST.COM for oryx/127.12.138.253@KRBTEST.COM
I20260811 02:52:22.622771 12843 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDawEwS/build/debug/bin/kudu
/tmp/dist-test-taskDawEwS/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786416707705494-12843-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786416707705494-12843-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786416707705494-12843-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786416707705494-12843-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.138.195:42681
--local_ip_for_outbound_sockets=127.12.138.195
--tserver_master_addrs=127.12.138.254:43457
--webserver_port=42669
--webserver_interface=127.12.138.195
--keytab_file=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786416707705494-12843-0/krb5kdc/oryx_127.12.138.195.keytab
--principal=oryx/127.12.138.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.138.212:41877
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.12.138.254:43457,127.12.138.253:42467
--tserver_master_addrs=127.12.138.254:43457,127.12.138.253:42467,127.12.138.252:42917 with env {KRB5CCNAME=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786416707705494-12843-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786416707705494-12843-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786416707705494-12843-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 02:52:22.623945 20977 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:52:22.588170 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.138.194:47333 (local address 127.12.138.253:42467)
0811 02:52:22.588350 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0811 02:52:22.588355 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 02:52:22.589872 (+ 1517us) server_negotiation.cc:407] Connection header received
0811 02:52:22.590925 (+ 1053us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:52:22.590928 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:52:22.590984 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:52:22.591040 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:52:22.592440 (+ 1400us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:52:22.593279 (+ 839us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:52:22.615200 (+ 21921us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:52:22.615737 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:52:22.615933 (+ 196us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:52:22.620229 (+ 4296us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:52:22.620232 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:52:22.620235 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:52:22.620264 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:52:22.621943 (+ 1679us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:52:22.622496 (+ 553us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:52:22.622499 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:52:22.622501 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:52:22.622591 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:52:22.622944 (+ 353us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:52:22.622949 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:52:22.622952 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:52:22.623200 (+ 248us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:52:22.623290 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0811 02:52:22.623713 (+ 423us) server_negotiation.cc:299] Negotiation successful
0811 02:52:22.623827 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82}
I20260811 02:52:22.624091 21424 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:52:22.588619 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.138.253:42467 (local address 127.12.138.194:47333)
0811 02:52:22.589760 (+ 1141us) negotiation.cc:107] Waiting for socket to connect
0811 02:52:22.589773 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 02:52:22.590685 (+ 912us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:52:22.591247 (+ 562us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:52:22.591255 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:52:22.591771 (+ 516us) client_negotiation.cc:189] Negotiated authn=SASL
0811 02:52:22.592267 (+ 496us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:52:22.592284 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:52:22.593432 (+ 1148us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:52:22.593434 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:52:22.594239 (+ 805us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:52:22.594247 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:52:22.616459 (+ 22212us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:52:22.616467 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:52:22.616611 (+ 144us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:52:22.618031 (+ 1420us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 02:52:22.618051 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0811 02:52:22.620039 (+ 1988us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 02:52:22.622084 (+ 2045us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:52:22.622087 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:52:22.622089 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:52:22.622372 (+ 283us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:52:22.622701 (+ 329us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:52:22.622706 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:52:22.622708 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:52:22.622816 (+ 108us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:52:22.623315 (+ 499us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 02:52:22.623321 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 02:52:22.623447 (+ 126us) client_negotiation.cc:769] Sending connection context
0811 02:52:22.623562 (+ 115us) client_negotiation.cc:240] Negotiation successful
0811 02:52:22.623783 (+ 221us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1058,"mutex_wait_us":222,"spinlock_wait_cycles":15488,"thread_start_us":181,"threads_started":1}
I20260811 02:52:22.624814 21420 heartbeater.cc:347] Connected to a master server at 127.12.138.253:42467
I20260811 02:52:22.624939 21420 heartbeater.cc:464] Registering TS with master...
I20260811 02:52:22.625229 21420 heartbeater.cc:511] Master 127.12.138.253:42467 requested a full tablet report, sending...
I20260811 02:52:22.626055 20830 ts_manager.cc:194] Registered new tserver with Master: c51f9680930c4419a91d0d0f521006dd (127.12.138.194:40543)
I20260811 02:52:22.740774 21266 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:52:22.736664 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34400 (local address 127.12.138.252:42917)
0811 02:52:22.736815 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0811 02:52:22.736821 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 02:52:22.736948 (+ 127us) server_negotiation.cc:407] Connection header received
0811 02:52:22.737060 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:52:22.737064 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:52:22.737135 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:52:22.737273 (+ 138us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 02:52:22.737510 (+ 237us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:52:22.738316 (+ 806us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:52:22.739418 (+ 1102us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:52:22.740106 (+ 688us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:52:22.740247 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:52:22.740413 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0811 02:52:22.740532 (+ 119us) server_negotiation.cc:299] Negotiation successful
0811 02:52:22.740609 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260811 02:52:22.740859 21428 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:52:22.736528 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.138.252:42917 (local address 127.0.0.1:34400)
0811 02:52:22.736850 (+ 322us) negotiation.cc:107] Waiting for socket to connect
0811 02:52:22.736863 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 02:52:22.736975 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:52:22.737285 (+ 310us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:52:22.737287 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:52:22.737306 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 02:52:22.737405 (+ 99us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:52:22.737410 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:52:22.738448 (+ 1038us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:52:22.738451 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:52:22.739243 (+ 792us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:52:22.739253 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:52:22.740297 (+ 1044us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:52:22.740308 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:52:22.740395 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:52:22.740418 (+ 23us) client_negotiation.cc:769] Sending connection context
0811 02:52:22.740508 (+ 90us) client_negotiation.cc:240] Negotiation successful
0811 02:52:22.740580 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":245,"thread_start_us":82,"threads_started":1}
I20260811 02:52:22.741559 21069 raft_consensus.cc:1203] T 00000000000000000000000000000000 P 3b8a3f24429b46a29f1697d97b2da6d1 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.6->[1.7-1.7] Dedup: 1.7->[]
I20260811 02:52:22.742058 21432 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3b8a3f24429b46a29f1697d97b2da6d1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0317e693c61849e9bf42d55fbf6ef321. Latest consensus state: current_term: 1 leader_uuid: "0317e693c61849e9bf42d55fbf6ef321" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "0317e693c61849e9bf42d55fbf6ef321" member_type: VOTER last_known_addr { host: "127.12.138.254" port: 43457 } } peers { permanent_uuid: "269d7f529b994b32bc6d6d1b512371d4" member_type: VOTER last_known_addr { host: "127.12.138.253" port: 42467 } attrs { promote: false } } peers { permanent_uuid: "3b8a3f24429b46a29f1697d97b2da6d1" member_type: NON_VOTER last_known_addr { host: "127.12.138.252" port: 42917 } attrs { promote: true } } }
I20260811 02:52:22.742210 21432 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3b8a3f24429b46a29f1697d97b2da6d1 [sys.catalog]: This master's current role is: LEARNER
W20260811 02:52:22.743952 21426 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 02:52:22.744268 21426 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 02:52:22.744367 21426 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 02:52:22.748603 21426 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 02:52:22.748667 21426 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 02:52:22.748793 21426 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.138.195
I20260811 02:52:22.754289 21426 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.138.212:41877
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786416707705494-12843-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786416707705494-12843-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786416707705494-12843-0/krb5kdc/oryx_127.12.138.195.keytab
--principal=oryx/127.12.138.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.138.195:42681
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786416707705494-12843-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.12.138.195
--webserver_port=42669
--tserver_master_addrs=127.12.138.254:43457,127.12.138.253:42467,127.12.138.252:42917
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21426
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.138.195
--log_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786416707705494-12843-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 02:09:49 UTC on ccycloud.quasar-fsnqqc.root.comops.site
build id 4579570
I20260811 02:52:22.755411 21426 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 02:52:22.756512 21426 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 02:52:22.766767 21437 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:52:22.766806 21434 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:52:22.766767 21435 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:52:22.767205 21426 server_base.cc:1034] running on GCE node
I20260811 02:52:22.767637 21426 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 02:52:22.768366 21426 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 02:52:22.769548 21426 hybrid_clock.cc:648] HybridClock initialized: now 1786416742769515 us; error 46 us; skew 500 ppm
Aug 11 02:52:22 dist-test-slave-9gcw krb5kdc[20077](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416742, etypes {rep=17 tkt=17 ses=17}, oryx/127.12.138.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 02:52:22.773017 21426 init.cc:377] Logged in from keytab as oryx/127.12.138.195@KRBTEST.COM (short username oryx)
I20260811 02:52:22.774106 21426 webserver.cc:460] Webserver started at http://127.12.138.195:42669/ using document root <none> and password file <none>
I20260811 02:52:22.774770 21426 fs_manager.cc:359] Metadata directory not provided
I20260811 02:52:22.774897 21426 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 02:52:22.779546 21426 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260811 02:52:22.781980 21444 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 02:52:22.783313 21426 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260811 02:52:22.783483 21426 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786416707705494-12843-0/minicluster-data/ts-2/data,/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786416707705494-12843-0/minicluster-data/ts-2/wal
uuid: "adde0ab9117a451c8cf06b46870c7c07"
format_stamp: "Formatted at 2026-08-11 02:52:21 on dist-test-slave-9gcw"
server_key: "99d255acabd2ed71a7cc41699f59754f"
server_key_iv: "04aaf406d1962249d1a260030a7b1add"
server_key_version: "encryptionkey@0"
I20260811 02:52:22.784152 21426 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786416707705494-12843-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786416707705494-12843-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786416707705494-12843-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 02:52:22.798887 21426 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 02:52:22.802659 21426 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 02:52:22.802891 21426 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 02:52:22.803525 21426 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 02:52:22.804533 21426 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 02:52:22.804605 21426 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 02:52:22.804668 21426 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 02:52:22.804706 21426 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 02:52:22.815378 21426 rpc_server.cc:307] RPC server started. Bound to: 127.12.138.195:42681
I20260811 02:52:22.815402 21572 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.138.195:42681 every 8 connection(s)
I20260811 02:52:22.816728 21426 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786416707705494-12843-0/minicluster-data/ts-2/data/info.pb
I20260811 02:52:22.821017 12843 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDawEwS/build/debug/bin/kudu as pid 21426
Aug 11 02:52:22 dist-test-slave-9gcw krb5kdc[20077](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416742, etypes {rep=17 tkt=17 ses=17}, oryx/127.12.138.195@KRBTEST.COM for oryx/127.12.138.253@KRBTEST.COM
Aug 11 02:52:22 dist-test-slave-9gcw krb5kdc[20077](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416742, etypes {rep=17 tkt=17 ses=17}, oryx/127.12.138.195@KRBTEST.COM for oryx/127.12.138.254@KRBTEST.COM
Aug 11 02:52:22 dist-test-slave-9gcw krb5kdc[20077](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416742, etypes {rep=17 tkt=17 ses=17}, oryx/127.12.138.195@KRBTEST.COM for oryx/127.12.138.252@KRBTEST.COM
I20260811 02:52:22.836370 21591 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:52:22.824848 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34406 (local address 127.12.138.252:42917)
0811 02:52:22.825432 (+ 584us) server_negotiation.cc:206] Beginning negotiation
0811 02:52:22.825436 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 02:52:22.826050 (+ 614us) server_negotiation.cc:407] Connection header received
0811 02:52:22.826089 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:52:22.826091 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:52:22.826136 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:52:22.826212 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:52:22.827276 (+ 1064us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:52:22.828061 (+ 785us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:52:22.829322 (+ 1261us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:52:22.829695 (+ 373us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:52:22.829804 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:52:22.831764 (+ 1960us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:52:22.831768 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:52:22.831771 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:52:22.831800 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:52:22.833885 (+ 2085us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:52:22.834616 (+ 731us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:52:22.834632 (+ 16us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:52:22.834634 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:52:22.834729 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:52:22.835076 (+ 347us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:52:22.835079 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:52:22.835081 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:52:22.835313 (+ 232us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:52:22.835465 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0811 02:52:22.836048 (+ 583us) server_negotiation.cc:299] Negotiation successful
0811 02:52:22.836191 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":507,"thread_start_us":83,"threads_started":1}
I20260811 02:52:22.838694 21266 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:52:22.819452 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.138.195:49455 (local address 127.12.138.252:42917)
0811 02:52:22.819597 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0811 02:52:22.819602 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 02:52:22.820136 (+ 534us) server_negotiation.cc:407] Connection header received
0811 02:52:22.821408 (+ 1272us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:52:22.821412 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:52:22.821474 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:52:22.821528 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:52:22.823829 (+ 2301us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:52:22.824580 (+ 751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:52:22.827854 (+ 3274us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:52:22.828256 (+ 402us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:52:22.828338 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:52:22.833196 (+ 4858us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:52:22.833200 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:52:22.833203 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:52:22.833238 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:52:22.834971 (+ 1733us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:52:22.835709 (+ 738us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:52:22.835713 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:52:22.835716 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:52:22.835813 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:52:22.836436 (+ 623us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:52:22.836440 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:52:22.836441 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:52:22.836682 (+ 241us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:52:22.836774 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0811 02:52:22.838417 (+ 1643us) server_negotiation.cc:299] Negotiation successful
0811 02:52:22.838545 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260811 02:52:22.839572 20182 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:52:22.818895 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.138.195:39873 (local address 127.12.138.254:43457)
0811 02:52:22.819223 (+ 328us) server_negotiation.cc:206] Beginning negotiation
0811 02:52:22.819229 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 02:52:22.820792 (+ 1563us) server_negotiation.cc:407] Connection header received
0811 02:52:22.821408 (+ 616us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:52:22.821412 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:52:22.821471 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:52:22.821528 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:52:22.823819 (+ 2291us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:52:22.824649 (+ 830us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:52:22.826688 (+ 2039us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:52:22.827082 (+ 394us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:52:22.827207 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:52:22.833103 (+ 5896us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:52:22.833106 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:52:22.833109 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:52:22.833130 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:52:22.837230 (+ 4100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:52:22.837775 (+ 545us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:52:22.837779 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:52:22.837780 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:52:22.837880 (+ 100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:52:22.838216 (+ 336us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:52:22.838220 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:52:22.838222 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:52:22.838462 (+ 240us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:52:22.838556 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0811 02:52:22.839293 (+ 737us) server_negotiation.cc:299] Negotiation successful
0811 02:52:22.839433 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":221}
I20260811 02:52:22.840698 21579 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:52:22.819223 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.138.254:43457 (local address 127.12.138.195:39873)
0811 02:52:22.820558 (+ 1335us) negotiation.cc:107] Waiting for socket to connect
0811 02:52:22.820571 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 02:52:22.821162 (+ 591us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:52:22.821953 (+ 791us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:52:22.821963 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:52:22.822748 (+ 785us) client_negotiation.cc:189] Negotiated authn=SASL
0811 02:52:22.823634 (+ 886us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:52:22.823658 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:52:22.824805 (+ 1147us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:52:22.824808 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:52:22.826531 (+ 1723us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:52:22.826543 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:52:22.827349 (+ 806us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:52:22.827353 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:52:22.827451 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:52:22.828497 (+ 1046us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 02:52:22.828516 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0811 02:52:22.832975 (+ 4459us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 02:52:22.837337 (+ 4362us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:52:22.837340 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:52:22.837342 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:52:22.837656 (+ 314us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:52:22.837993 (+ 337us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:52:22.837996 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:52:22.837998 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:52:22.838092 (+ 94us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:52:22.838603 (+ 511us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 02:52:22.838606 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 02:52:22.838693 (+ 87us) client_negotiation.cc:769] Sending connection context
0811 02:52:22.839091 (+ 398us) client_negotiation.cc:240] Negotiation successful
0811 02:52:22.839267 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1253,"mutex_wait_us":1474,"spinlock_wait_cycles":3328,"thread_start_us":159,"threads_started":1}
I20260811 02:52:22.840972 21577 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:52:22.819223 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.138.252:42917 (local address 127.12.138.195:49455)
0811 02:52:22.819958 (+ 735us) negotiation.cc:107] Waiting for socket to connect
0811 02:52:22.820024 (+ 66us) client_negotiation.cc:174] Beginning negotiation
0811 02:52:22.821141 (+ 1117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:52:22.821953 (+ 812us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:52:22.821963 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:52:22.822750 (+ 787us) client_negotiation.cc:189] Negotiated authn=SASL
0811 02:52:22.823635 (+ 885us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:52:22.823657 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:52:22.824801 (+ 1144us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:52:22.824804 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:52:22.827703 (+ 2899us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:52:22.827713 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:52:22.829243 (+ 1530us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:52:22.829247 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:52:22.829382 (+ 135us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:52:22.829949 (+ 567us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 02:52:22.829964 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0811 02:52:22.832671 (+ 2707us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 02:52:22.835119 (+ 2448us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:52:22.835127 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:52:22.835145 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:52:22.835576 (+ 431us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:52:22.836083 (+ 507us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:52:22.836086 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:52:22.836088 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:52:22.836306 (+ 218us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:52:22.837690 (+ 1384us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 02:52:22.837700 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 02:52:22.838087 (+ 387us) client_negotiation.cc:769] Sending connection context
0811 02:52:22.838862 (+ 775us) client_negotiation.cc:240] Negotiation successful
0811 02:52:22.839860 (+ 998us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":456,"mutex_wait_us":2301,"spinlock_wait_cycles":33792,"thread_start_us":696,"threads_started":1}
I20260811 02:52:22.841952 21589 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:52:22.824184 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39466 (local address 127.12.138.254:43457)
0811 02:52:22.827394 (+ 3210us) server_negotiation.cc:206] Beginning negotiation
0811 02:52:22.827399 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 02:52:22.827410 (+ 11us) server_negotiation.cc:407] Connection header received
0811 02:52:22.827446 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:52:22.827449 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:52:22.827492 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:52:22.827590 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:52:22.828567 (+ 977us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:52:22.829437 (+ 870us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:52:22.830760 (+ 1323us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:52:22.830990 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:52:22.831122 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:52:22.832926 (+ 1804us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:52:22.832932 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:52:22.832939 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:52:22.832973 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:52:22.839441 (+ 6468us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:52:22.840015 (+ 574us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:52:22.840019 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:52:22.840020 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:52:22.840118 (+ 98us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:52:22.840458 (+ 340us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:52:22.840462 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:52:22.840464 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:52:22.840770 (+ 306us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:52:22.840908 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0811 02:52:22.841681 (+ 773us) server_negotiation.cc:299] Negotiation successful
0811 02:52:22.841802 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3141,"thread_start_us":81,"threads_started":1}
I20260811 02:52:22.841943 20977 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:52:22.819053 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.138.195:57525 (local address 127.12.138.253:42467)
0811 02:52:22.819337 (+ 284us) server_negotiation.cc:206] Beginning negotiation
0811 02:52:22.819344 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 02:52:22.820381 (+ 1037us) server_negotiation.cc:407] Connection header received
0811 02:52:22.821449 (+ 1068us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:52:22.821452 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:52:22.821523 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:52:22.821582 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:52:22.823819 (+ 2237us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:52:22.824659 (+ 840us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:52:22.826693 (+ 2034us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:52:22.826923 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:52:22.827043 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:52:22.834599 (+ 7556us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:52:22.834604 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:52:22.834608 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:52:22.834640 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:52:22.836452 (+ 1812us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:52:22.839791 (+ 3339us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:52:22.839797 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:52:22.839801 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:52:22.839901 (+ 100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:52:22.840913 (+ 1012us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:52:22.840918 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:52:22.840921 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:52:22.841190 (+ 269us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:52:22.841279 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0811 02:52:22.841667 (+ 388us) server_negotiation.cc:299] Negotiation successful
0811 02:52:22.841794 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":200}
I20260811 02:52:22.842428 21574 heartbeater.cc:347] Connected to a master server at 127.12.138.254:43457
I20260811 02:52:22.842489 21578 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:52:22.819223 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.138.253:42467 (local address 127.12.138.195:57525)
0811 02:52:22.820233 (+ 1010us) negotiation.cc:107] Waiting for socket to connect
0811 02:52:22.820252 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0811 02:52:22.821259 (+ 1007us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:52:22.821955 (+ 696us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:52:22.821964 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:52:22.822970 (+ 1006us) client_negotiation.cc:189] Negotiated authn=SASL
0811 02:52:22.823634 (+ 664us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:52:22.823658 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:52:22.824805 (+ 1147us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:52:22.824808 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:52:22.826526 (+ 1718us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:52:22.826539 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:52:22.827094 (+ 555us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:52:22.827098 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:52:22.827303 (+ 205us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:52:22.828163 (+ 860us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 02:52:22.828189 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0811 02:52:22.834430 (+ 6241us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 02:52:22.837363 (+ 2933us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:52:22.837366 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:52:22.837369 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:52:22.839153 (+ 1784us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:52:22.840024 (+ 871us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:52:22.840027 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:52:22.840030 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:52:22.840116 (+ 86us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:52:22.841339 (+ 1223us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 02:52:22.841342 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 02:52:22.841449 (+ 107us) client_negotiation.cc:769] Sending connection context
0811 02:52:22.842163 (+ 714us) client_negotiation.cc:240] Negotiation successful
0811 02:52:22.842322 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":907,"mutex_wait_us":249,"spinlock_wait_cycles":4736,"thread_start_us":174,"threads_started":1}
I20260811 02:52:22.842428 21573 heartbeater.cc:347] Connected to a master server at 127.12.138.252:42917
I20260811 02:52:22.842677 21574 heartbeater.cc:464] Registering TS with master...
I20260811 02:52:22.843127 21575 heartbeater.cc:347] Connected to a master server at 127.12.138.253:42467
I20260811 02:52:22.843231 21575 heartbeater.cc:464] Registering TS with master...
I20260811 02:52:22.843329 21574 heartbeater.cc:511] Master 127.12.138.254:43457 requested a full tablet report, sending...
I20260811 02:52:22.842677 21573 heartbeater.cc:464] Registering TS with master...
I20260811 02:52:22.843832 21575 heartbeater.cc:511] Master 127.12.138.253:42467 requested a full tablet report, sending...
I20260811 02:52:22.843937 21573 heartbeater.cc:511] Master 127.12.138.252:42917 requested a full tablet report, sending...
I20260811 02:52:22.844439 20136 ts_manager.cc:194] Re-registered known tserver with Master: adde0ab9117a451c8cf06b46870c7c07 (127.12.138.195:42681)
I20260811 02:52:22.844661 21056 ts_manager.cc:194] Registered new tserver with Master: adde0ab9117a451c8cf06b46870c7c07 (127.12.138.195:42681)
I20260811 02:52:22.844802 20830 ts_manager.cc:194] Registered new tserver with Master: adde0ab9117a451c8cf06b46870c7c07 (127.12.138.195:42681)
I20260811 02:52:22.845197 20136 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.12.138.195@KRBTEST.COM'} at 127.12.138.195:39873
I20260811 02:52:22.848400 21590 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:52:22.824457 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44228 (local address 127.12.138.253:42467)
0811 02:52:22.831416 (+ 6959us) server_negotiation.cc:206] Beginning negotiation
0811 02:52:22.831423 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 02:52:22.831436 (+ 13us) server_negotiation.cc:407] Connection header received
0811 02:52:22.831475 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:52:22.831478 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:52:22.831531 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:52:22.831615 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:52:22.833227 (+ 1612us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:52:22.834123 (+ 896us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:52:22.836098 (+ 1975us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:52:22.836443 (+ 345us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:52:22.836545 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:52:22.839289 (+ 2744us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:52:22.839297 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:52:22.839302 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:52:22.839332 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:52:22.846377 (+ 7045us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:52:22.846935 (+ 558us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:52:22.846940 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:52:22.846942 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:52:22.847068 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:52:22.847419 (+ 351us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:52:22.847424 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:52:22.847426 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:52:22.847626 (+ 200us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:52:22.847780 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0811 02:52:22.848057 (+ 277us) server_negotiation.cc:299] Negotiation successful
0811 02:52:22.848202 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":6801,"thread_start_us":82,"threads_started":1}
I20260811 02:52:22.845955 20135 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39466:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 02:52:22.850008 20135 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 02:52:22.852834 21427 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0317e693c61849e9bf42d55fbf6ef321 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "0317e693c61849e9bf42d55fbf6ef321" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "0317e693c61849e9bf42d55fbf6ef321" member_type: VOTER last_known_addr { host: "127.12.138.254" port: 43457 } } peers { permanent_uuid: "269d7f529b994b32bc6d6d1b512371d4" member_type: VOTER last_known_addr { host: "127.12.138.253" port: 42467 } attrs { promote: false } } peers { permanent_uuid: "3b8a3f24429b46a29f1697d97b2da6d1" member_type: NON_VOTER last_known_addr { host: "127.12.138.252" port: 42917 } attrs { promote: true } } }
I20260811 02:52:22.852984 21427 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0317e693c61849e9bf42d55fbf6ef321 [sys.catalog]: This master's current role is: LEADER
I20260811 02:52:22.854800 21427 raft_consensus.cc:1050] T 00000000000000000000000000000000 P 0317e693c61849e9bf42d55fbf6ef321: attempting to promote NON_VOTER 3b8a3f24429b46a29f1697d97b2da6d1 to VOTER
I20260811 02:52:22.855172 21427 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0317e693c61849e9bf42d55fbf6ef321 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 1.8, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "0317e693c61849e9bf42d55fbf6ef321" member_type: VOTER last_known_addr { host: "127.12.138.254" port: 43457 } } peers { permanent_uuid: "269d7f529b994b32bc6d6d1b512371d4" member_type: VOTER last_known_addr { host: "127.12.138.253" port: 42467 } attrs { promote: false } } peers { permanent_uuid: "3b8a3f24429b46a29f1697d97b2da6d1" member_type: VOTER last_known_addr { host: "127.12.138.252" port: 42917 } attrs { promote: false } }
I20260811 02:52:22.856390 21069 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 3b8a3f24429b46a29f1697d97b2da6d1 [term 1 LEARNER]: Refusing update from remote peer 0317e693c61849e9bf42d55fbf6ef321: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20260811 02:52:22.856390 20849 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 269d7f529b994b32bc6d6d1b512371d4 [term 1 FOLLOWER]: Refusing update from remote peer 0317e693c61849e9bf42d55fbf6ef321: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20260811 02:52:22.856896 21427 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 0317e693c61849e9bf42d55fbf6ef321 [LEADER]: Connected to new peer: Peer: permanent_uuid: "269d7f529b994b32bc6d6d1b512371d4" member_type: VOTER last_known_addr { host: "127.12.138.253" port: 42467 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.000s
I20260811 02:52:22.857023 21594 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 0317e693c61849e9bf42d55fbf6ef321 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b8a3f24429b46a29f1697d97b2da6d1" member_type: VOTER last_known_addr { host: "127.12.138.252" port: 42917 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.000s
I20260811 02:52:22.859022 21427 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 0317e693c61849e9bf42d55fbf6ef321 [term 1 LEADER]: Committing config change with OpId 1.9: config changed from index 7 to 9, 3b8a3f24429b46a29f1697d97b2da6d1 (127.12.138.252) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "0317e693c61849e9bf42d55fbf6ef321" member_type: VOTER last_known_addr { host: "127.12.138.254" port: 43457 } } peers { permanent_uuid: "269d7f529b994b32bc6d6d1b512371d4" member_type: VOTER last_known_addr { host: "127.12.138.253" port: 42467 } attrs { promote: false } } peers { permanent_uuid: "3b8a3f24429b46a29f1697d97b2da6d1" member_type: VOTER last_known_addr { host: "127.12.138.252" port: 42917 } attrs { promote: false } } }
I20260811 02:52:22.859644 21069 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 3b8a3f24429b46a29f1697d97b2da6d1 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 7 to 9, 3b8a3f24429b46a29f1697d97b2da6d1 (127.12.138.252) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "0317e693c61849e9bf42d55fbf6ef321" member_type: VOTER last_known_addr { host: "127.12.138.254" port: 43457 } } peers { permanent_uuid: "269d7f529b994b32bc6d6d1b512371d4" member_type: VOTER last_known_addr { host: "127.12.138.253" port: 42467 } attrs { promote: false } } peers { permanent_uuid: "3b8a3f24429b46a29f1697d97b2da6d1" member_type: VOTER last_known_addr { host: "127.12.138.252" port: 42917 } attrs { promote: false } } }
I20260811 02:52:22.861636 21432 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3b8a3f24429b46a29f1697d97b2da6d1 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "0317e693c61849e9bf42d55fbf6ef321" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "0317e693c61849e9bf42d55fbf6ef321" member_type: VOTER last_known_addr { host: "127.12.138.254" port: 43457 } } peers { permanent_uuid: "269d7f529b994b32bc6d6d1b512371d4" member_type: VOTER last_known_addr { host: "127.12.138.253" port: 42467 } attrs { promote: false } } peers { permanent_uuid: "3b8a3f24429b46a29f1697d97b2da6d1" member_type: VOTER last_known_addr { host: "127.12.138.252" port: 42917 } attrs { promote: false } } }
I20260811 02:52:22.861783 21432 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3b8a3f24429b46a29f1697d97b2da6d1 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 02:52:22.861949 21595 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0317e693c61849e9bf42d55fbf6ef321 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "0317e693c61849e9bf42d55fbf6ef321" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "0317e693c61849e9bf42d55fbf6ef321" member_type: VOTER last_known_addr { host: "127.12.138.254" port: 43457 } } peers { permanent_uuid: "269d7f529b994b32bc6d6d1b512371d4" member_type: VOTER last_known_addr { host: "127.12.138.253" port: 42467 } attrs { promote: false } } peers { permanent_uuid: "3b8a3f24429b46a29f1697d97b2da6d1" member_type: VOTER last_known_addr { host: "127.12.138.252" port: 42917 } attrs { promote: false } } }
I20260811 02:52:22.862066 21595 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0317e693c61849e9bf42d55fbf6ef321 [sys.catalog]: This master's current role is: LEADER
I20260811 02:52:22.862707 21595 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0317e693c61849e9bf42d55fbf6ef321 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "0317e693c61849e9bf42d55fbf6ef321" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "0317e693c61849e9bf42d55fbf6ef321" member_type: VOTER last_known_addr { host: "127.12.138.254" port: 43457 } } peers { permanent_uuid: "269d7f529b994b32bc6d6d1b512371d4" member_type: VOTER last_known_addr { host: "127.12.138.253" port: 42467 } attrs { promote: false } } peers { permanent_uuid: "3b8a3f24429b46a29f1697d97b2da6d1" member_type: VOTER last_known_addr { host: "127.12.138.252" port: 42917 } attrs { promote: false } } }
I20260811 02:52:22.862823 21595 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0317e693c61849e9bf42d55fbf6ef321 [sys.catalog]: This master's current role is: LEADER
I20260811 02:52:22.862708 20849 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 269d7f529b994b32bc6d6d1b512371d4 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 7 to 9, 3b8a3f24429b46a29f1697d97b2da6d1 (127.12.138.252) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "0317e693c61849e9bf42d55fbf6ef321" member_type: VOTER last_known_addr { host: "127.12.138.254" port: 43457 } } peers { permanent_uuid: "269d7f529b994b32bc6d6d1b512371d4" member_type: VOTER last_known_addr { host: "127.12.138.253" port: 42467 } attrs { promote: false } } peers { permanent_uuid: "3b8a3f24429b46a29f1697d97b2da6d1" member_type: VOTER last_known_addr { host: "127.12.138.252" port: 42917 } attrs { promote: false } } }
I20260811 02:52:22.862630 21427 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0317e693c61849e9bf42d55fbf6ef321 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "0317e693c61849e9bf42d55fbf6ef321" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "0317e693c61849e9bf42d55fbf6ef321" member_type: VOTER last_known_addr { host: "127.12.138.254" port: 43457 } } peers { permanent_uuid: "269d7f529b994b32bc6d6d1b512371d4" member_type: VOTER last_known_addr { host: "127.12.138.253" port: 42467 } attrs { promote: false } } peers { permanent_uuid: "3b8a3f24429b46a29f1697d97b2da6d1" member_type: VOTER last_known_addr { host: "127.12.138.252" port: 42917 } attrs { promote: false } } }
I20260811 02:52:22.863010 21427 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0317e693c61849e9bf42d55fbf6ef321 [sys.catalog]: This master's current role is: LEADER
I20260811 02:52:22.864723 21603 sys_catalog.cc:455] T 00000000000000000000000000000000 P 269d7f529b994b32bc6d6d1b512371d4 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "0317e693c61849e9bf42d55fbf6ef321" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "0317e693c61849e9bf42d55fbf6ef321" member_type: VOTER last_known_addr { host: "127.12.138.254" port: 43457 } } peers { permanent_uuid: "269d7f529b994b32bc6d6d1b512371d4" member_type: VOTER last_known_addr { host: "127.12.138.253" port: 42467 } attrs { promote: false } } peers { permanent_uuid: "3b8a3f24429b46a29f1697d97b2da6d1" member_type: VOTER last_known_addr { host: "127.12.138.252" port: 42917 } attrs { promote: false } } }
I20260811 02:52:22.864938 21603 sys_catalog.cc:458] T 00000000000000000000000000000000 P 269d7f529b994b32bc6d6d1b512371d4 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 02:52:22.875566 21428 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:52:22.869444 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.138.193:41461 (local address 127.0.0.1:34046)
0811 02:52:22.869595 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0811 02:52:22.869612 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 02:52:22.869790 (+ 178us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:52:22.871060 (+ 1270us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:52:22.871062 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:52:22.871075 (+ 13us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 02:52:22.871199 (+ 124us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:52:22.871205 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:52:22.872698 (+ 1493us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:52:22.872701 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:52:22.873946 (+ 1245us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:52:22.873957 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:52:22.875130 (+ 1173us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:52:22.875133 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:52:22.875228 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:52:22.875257 (+ 29us) client_negotiation.cc:769] Sending connection context
0811 02:52:22.875302 (+ 45us) client_negotiation.cc:240] Negotiation successful
0811 02:52:22.875405 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":51}
I20260811 02:52:22.876886 21606 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:52:22.870047 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.138.195:42681 (local address 127.0.0.1:36444)
0811 02:52:22.870586 (+ 539us) negotiation.cc:107] Waiting for socket to connect
0811 02:52:22.870599 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 02:52:22.870681 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:52:22.872129 (+ 1448us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:52:22.872133 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:52:22.872152 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 02:52:22.872249 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:52:22.872255 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:52:22.873531 (+ 1276us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:52:22.873535 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:52:22.874763 (+ 1228us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:52:22.874773 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:52:22.876449 (+ 1676us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:52:22.876453 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:52:22.876549 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:52:22.876575 (+ 26us) client_negotiation.cc:769] Sending connection context
0811 02:52:22.876657 (+ 82us) client_negotiation.cc:240] Negotiation successful
0811 02:52:22.876735 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":485,"thread_start_us":100,"threads_started":1}
I20260811 02:52:22.877003 21605 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:52:22.869659 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34046 (local address 127.12.138.193:41461)
0811 02:52:22.870049 (+ 390us) server_negotiation.cc:206] Beginning negotiation
0811 02:52:22.870069 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0811 02:52:22.870097 (+ 28us) server_negotiation.cc:407] Connection header received
0811 02:52:22.870651 (+ 554us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:52:22.870661 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:52:22.870934 (+ 273us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:52:22.871091 (+ 157us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 02:52:22.871345 (+ 254us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:52:22.872578 (+ 1233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:52:22.874100 (+ 1522us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:52:22.874980 (+ 880us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:52:22.876560 (+ 1580us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:52:22.876691 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0811 02:52:22.876742 (+ 51us) server_negotiation.cc:299] Negotiation successful
0811 02:52:22.876834 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":281,"thread_start_us":115,"threads_started":1}
I20260811 02:52:22.877624 21607 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:52:22.870210 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36444 (local address 127.12.138.195:42681)
0811 02:52:22.871434 (+ 1224us) server_negotiation.cc:206] Beginning negotiation
0811 02:52:22.871462 (+ 28us) server_negotiation.cc:399] Waiting for connection header
0811 02:52:22.871487 (+ 25us) server_negotiation.cc:407] Connection header received
0811 02:52:22.871567 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:52:22.871572 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:52:22.871791 (+ 219us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:52:22.871974 (+ 183us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 02:52:22.872382 (+ 408us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:52:22.873400 (+ 1018us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:52:22.874920 (+ 1520us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:52:22.876089 (+ 1169us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:52:22.876229 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:52:22.876383 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0811 02:52:22.877341 (+ 958us) server_negotiation.cc:299] Negotiation successful
0811 02:52:22.877450 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1102,"thread_start_us":162,"threads_started":1}
I20260811 02:52:22.880221 21185 tablet_service.cc:1467] Processing CreateTablet for tablet 217c6747ae2c4516817d2f549f1756ed (DEFAULT_TABLE table=test-table [id=6fcef6450ac74a2f996968e2efd54757]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 02:52:22.880963 21499 tablet_service.cc:1467] Processing CreateTablet for tablet 217c6747ae2c4516817d2f549f1756ed (DEFAULT_TABLE table=test-table [id=6fcef6450ac74a2f996968e2efd54757]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 02:52:22.881618 21185 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 217c6747ae2c4516817d2f549f1756ed. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 02:52:22.882071 21499 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 217c6747ae2c4516817d2f549f1756ed. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 02:52:22.886232 21608 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:52:22.870354 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.138.194:40543 (local address 127.0.0.1:52766)
0811 02:52:22.870834 (+ 480us) negotiation.cc:107] Waiting for socket to connect
0811 02:52:22.870846 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 02:52:22.870915 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:52:22.874379 (+ 3464us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:52:22.874383 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:52:22.874400 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 02:52:22.874514 (+ 114us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:52:22.874520 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:52:22.876212 (+ 1692us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:52:22.876216 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:52:22.877548 (+ 1332us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:52:22.877560 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:52:22.885580 (+ 8020us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:52:22.885585 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:52:22.885771 (+ 186us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:52:22.885825 (+ 54us) client_negotiation.cc:769] Sending connection context
0811 02:52:22.885926 (+ 101us) client_negotiation.cc:240] Negotiation successful
0811 02:52:22.886033 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":432,"thread_start_us":66,"threads_started":1}
I20260811 02:52:22.886756 21609 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:52:22.870572 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52766 (local address 127.12.138.194:40543)
0811 02:52:22.873754 (+ 3182us) server_negotiation.cc:206] Beginning negotiation
0811 02:52:22.873766 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 02:52:22.873791 (+ 25us) server_negotiation.cc:407] Connection header received
0811 02:52:22.873869 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:52:22.873874 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:52:22.874071 (+ 197us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:52:22.874229 (+ 158us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 02:52:22.874647 (+ 418us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:52:22.876057 (+ 1410us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:52:22.877728 (+ 1671us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:52:22.885395 (+ 7667us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:52:22.885584 (+ 189us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:52:22.885746 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0811 02:52:22.885986 (+ 240us) server_negotiation.cc:299] Negotiation successful
0811 02:52:22.886100 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3051,"thread_start_us":134,"threads_started":1}
I20260811 02:52:22.890251 21610 tablet_bootstrap.cc:492] T 217c6747ae2c4516817d2f549f1756ed P adde0ab9117a451c8cf06b46870c7c07: Bootstrap starting.
I20260811 02:52:22.890178 21340 tablet_service.cc:1467] Processing CreateTablet for tablet 217c6747ae2c4516817d2f549f1756ed (DEFAULT_TABLE table=test-table [id=6fcef6450ac74a2f996968e2efd54757]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 02:52:22.891347 21340 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 217c6747ae2c4516817d2f549f1756ed. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 02:52:22.891294 21611 tablet_bootstrap.cc:492] T 217c6747ae2c4516817d2f549f1756ed P ab096c76c5544f828e9eb7f2bee1977e: Bootstrap starting.
I20260811 02:52:22.893433 21610 tablet_bootstrap.cc:654] T 217c6747ae2c4516817d2f549f1756ed P adde0ab9117a451c8cf06b46870c7c07: Neither blocks nor log segments found. Creating new log.
I20260811 02:52:22.893597 21611 tablet_bootstrap.cc:654] T 217c6747ae2c4516817d2f549f1756ed P ab096c76c5544f828e9eb7f2bee1977e: Neither blocks nor log segments found. Creating new log.
I20260811 02:52:22.894577 21610 log.cc:824] T 217c6747ae2c4516817d2f549f1756ed P adde0ab9117a451c8cf06b46870c7c07: Log is configured to *not* fsync() on all Append() calls
I20260811 02:52:22.894577 21611 log.cc:824] T 217c6747ae2c4516817d2f549f1756ed P ab096c76c5544f828e9eb7f2bee1977e: Log is configured to *not* fsync() on all Append() calls
I20260811 02:52:22.897367 21611 tablet_bootstrap.cc:492] T 217c6747ae2c4516817d2f549f1756ed P ab096c76c5544f828e9eb7f2bee1977e: No bootstrap required, opened a new log
I20260811 02:52:22.897481 21610 tablet_bootstrap.cc:492] T 217c6747ae2c4516817d2f549f1756ed P adde0ab9117a451c8cf06b46870c7c07: No bootstrap required, opened a new log
I20260811 02:52:22.897600 21611 ts_tablet_manager.cc:1397] T 217c6747ae2c4516817d2f549f1756ed P ab096c76c5544f828e9eb7f2bee1977e: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260811 02:52:22.897656 21610 ts_tablet_manager.cc:1397] T 217c6747ae2c4516817d2f549f1756ed P adde0ab9117a451c8cf06b46870c7c07: Time spent bootstrapping tablet: real 0.008s user 0.006s sys 0.000s
I20260811 02:52:22.899288 21615 tablet_bootstrap.cc:492] T 217c6747ae2c4516817d2f549f1756ed P c51f9680930c4419a91d0d0f521006dd: Bootstrap starting.
I20260811 02:52:22.901655 21615 tablet_bootstrap.cc:654] T 217c6747ae2c4516817d2f549f1756ed P c51f9680930c4419a91d0d0f521006dd: Neither blocks nor log segments found. Creating new log.
I20260811 02:52:22.902562 21615 log.cc:824] T 217c6747ae2c4516817d2f549f1756ed P c51f9680930c4419a91d0d0f521006dd: Log is configured to *not* fsync() on all Append() calls
I20260811 02:52:22.902688 21611 raft_consensus.cc:348] T 217c6747ae2c4516817d2f549f1756ed P ab096c76c5544f828e9eb7f2bee1977e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ab096c76c5544f828e9eb7f2bee1977e" member_type: VOTER last_known_addr { host: "127.12.138.193" port: 41461 } } peers { permanent_uuid: "adde0ab9117a451c8cf06b46870c7c07" member_type: VOTER last_known_addr { host: "127.12.138.195" port: 42681 } } peers { permanent_uuid: "c51f9680930c4419a91d0d0f521006dd" member_type: VOTER last_known_addr { host: "127.12.138.194" port: 40543 } }
I20260811 02:52:22.903048 21611 raft_consensus.cc:374] T 217c6747ae2c4516817d2f549f1756ed P ab096c76c5544f828e9eb7f2bee1977e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 02:52:22.903124 21611 raft_consensus.cc:729] T 217c6747ae2c4516817d2f549f1756ed P ab096c76c5544f828e9eb7f2bee1977e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ab096c76c5544f828e9eb7f2bee1977e, State: Initialized, Role: FOLLOWER
I20260811 02:52:22.903395 21610 raft_consensus.cc:348] T 217c6747ae2c4516817d2f549f1756ed P adde0ab9117a451c8cf06b46870c7c07 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ab096c76c5544f828e9eb7f2bee1977e" member_type: VOTER last_known_addr { host: "127.12.138.193" port: 41461 } } peers { permanent_uuid: "adde0ab9117a451c8cf06b46870c7c07" member_type: VOTER last_known_addr { host: "127.12.138.195" port: 42681 } } peers { permanent_uuid: "c51f9680930c4419a91d0d0f521006dd" member_type: VOTER last_known_addr { host: "127.12.138.194" port: 40543 } }
I20260811 02:52:22.903731 21610 raft_consensus.cc:374] T 217c6747ae2c4516817d2f549f1756ed P adde0ab9117a451c8cf06b46870c7c07 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 02:52:22.903640 21611 consensus_queue.cc:260] T 217c6747ae2c4516817d2f549f1756ed P ab096c76c5544f828e9eb7f2bee1977e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ab096c76c5544f828e9eb7f2bee1977e" member_type: VOTER last_known_addr { host: "127.12.138.193" port: 41461 } } peers { permanent_uuid: "adde0ab9117a451c8cf06b46870c7c07" member_type: VOTER last_known_addr { host: "127.12.138.195" port: 42681 } } peers { permanent_uuid: "c51f9680930c4419a91d0d0f521006dd" member_type: VOTER last_known_addr { host: "127.12.138.194" port: 40543 } }
I20260811 02:52:22.903803 21610 raft_consensus.cc:729] T 217c6747ae2c4516817d2f549f1756ed P adde0ab9117a451c8cf06b46870c7c07 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: adde0ab9117a451c8cf06b46870c7c07, State: Initialized, Role: FOLLOWER
I20260811 02:52:22.904315 21610 consensus_queue.cc:260] T 217c6747ae2c4516817d2f549f1756ed P adde0ab9117a451c8cf06b46870c7c07 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ab096c76c5544f828e9eb7f2bee1977e" member_type: VOTER last_known_addr { host: "127.12.138.193" port: 41461 } } peers { permanent_uuid: "adde0ab9117a451c8cf06b46870c7c07" member_type: VOTER last_known_addr { host: "127.12.138.195" port: 42681 } } peers { permanent_uuid: "c51f9680930c4419a91d0d0f521006dd" member_type: VOTER last_known_addr { host: "127.12.138.194" port: 40543 } }
I20260811 02:52:22.904413 21261 heartbeater.cc:503] Master 127.12.138.254:43457 was elected leader, sending a full tablet report...
I20260811 02:52:22.904811 21611 ts_tablet_manager.cc:1428] T 217c6747ae2c4516817d2f549f1756ed P ab096c76c5544f828e9eb7f2bee1977e: Time spent starting tablet: real 0.007s user 0.007s sys 0.000s
I20260811 02:52:22.905750 21574 heartbeater.cc:503] Master 127.12.138.254:43457 was elected leader, sending a full tablet report...
I20260811 02:52:22.906795 21610 ts_tablet_manager.cc:1428] T 217c6747ae2c4516817d2f549f1756ed P adde0ab9117a451c8cf06b46870c7c07: Time spent starting tablet: real 0.009s user 0.008s sys 0.000s
I20260811 02:52:22.911666 21615 tablet_bootstrap.cc:492] T 217c6747ae2c4516817d2f549f1756ed P c51f9680930c4419a91d0d0f521006dd: No bootstrap required, opened a new log
I20260811 02:52:22.911887 21615 ts_tablet_manager.cc:1397] T 217c6747ae2c4516817d2f549f1756ed P c51f9680930c4419a91d0d0f521006dd: Time spent bootstrapping tablet: real 0.013s user 0.004s sys 0.000s
I20260811 02:52:22.916267 21615 raft_consensus.cc:348] T 217c6747ae2c4516817d2f549f1756ed P c51f9680930c4419a91d0d0f521006dd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ab096c76c5544f828e9eb7f2bee1977e" member_type: VOTER last_known_addr { host: "127.12.138.193" port: 41461 } } peers { permanent_uuid: "adde0ab9117a451c8cf06b46870c7c07" member_type: VOTER last_known_addr { host: "127.12.138.195" port: 42681 } } peers { permanent_uuid: "c51f9680930c4419a91d0d0f521006dd" member_type: VOTER last_known_addr { host: "127.12.138.194" port: 40543 } }
I20260811 02:52:22.916595 21615 raft_consensus.cc:374] T 217c6747ae2c4516817d2f549f1756ed P c51f9680930c4419a91d0d0f521006dd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 02:52:22.916695 21615 raft_consensus.cc:729] T 217c6747ae2c4516817d2f549f1756ed P c51f9680930c4419a91d0d0f521006dd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c51f9680930c4419a91d0d0f521006dd, State: Initialized, Role: FOLLOWER
I20260811 02:52:22.917088 21615 consensus_queue.cc:260] T 217c6747ae2c4516817d2f549f1756ed P c51f9680930c4419a91d0d0f521006dd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ab096c76c5544f828e9eb7f2bee1977e" member_type: VOTER last_known_addr { host: "127.12.138.193" port: 41461 } } peers { permanent_uuid: "adde0ab9117a451c8cf06b46870c7c07" member_type: VOTER last_known_addr { host: "127.12.138.195" port: 42681 } } peers { permanent_uuid: "c51f9680930c4419a91d0d0f521006dd" member_type: VOTER last_known_addr { host: "127.12.138.194" port: 40543 } }
I20260811 02:52:22.917893 21419 heartbeater.cc:503] Master 127.12.138.254:43457 was elected leader, sending a full tablet report...
I20260811 02:52:22.918116 21615 ts_tablet_manager.cc:1428] T 217c6747ae2c4516817d2f549f1756ed P c51f9680930c4419a91d0d0f521006dd: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260811 02:52:22.918538 21617 raft_consensus.cc:482] T 217c6747ae2c4516817d2f549f1756ed P ab096c76c5544f828e9eb7f2bee1977e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 02:52:22.918699 21617 raft_consensus.cc:504] T 217c6747ae2c4516817d2f549f1756ed P ab096c76c5544f828e9eb7f2bee1977e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ab096c76c5544f828e9eb7f2bee1977e" member_type: VOTER last_known_addr { host: "127.12.138.193" port: 41461 } } peers { permanent_uuid: "adde0ab9117a451c8cf06b46870c7c07" member_type: VOTER last_known_addr { host: "127.12.138.195" port: 42681 } } peers { permanent_uuid: "c51f9680930c4419a91d0d0f521006dd" member_type: VOTER last_known_addr { host: "127.12.138.194" port: 40543 } }
I20260811 02:52:22.919986 21617 leader_election.cc:288] T 217c6747ae2c4516817d2f549f1756ed P ab096c76c5544f828e9eb7f2bee1977e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers adde0ab9117a451c8cf06b46870c7c07 (127.12.138.195:42681), c51f9680930c4419a91d0d0f521006dd (127.12.138.194:40543)
I20260811 02:52:22.926900 21609 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:52:22.921251 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.138.193:60261 (local address 127.12.138.194:40543)
0811 02:52:22.921366 (+ 115us) server_negotiation.cc:206] Beginning negotiation
0811 02:52:22.921372 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 02:52:22.921389 (+ 17us) server_negotiation.cc:407] Connection header received
0811 02:52:22.921532 (+ 143us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:52:22.921534 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:52:22.921597 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:52:22.921669 (+ 72us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 02:52:22.922262 (+ 593us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:52:22.922988 (+ 726us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:52:22.924586 (+ 1598us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:52:22.925006 (+ 420us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:52:22.925144 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:52:22.925234 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0811 02:52:22.926705 (+ 1471us) server_negotiation.cc:299] Negotiation successful
0811 02:52:22.926778 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260811 02:52:22.927002 21620 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:52:22.920903 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.138.194:40543 (local address 127.12.138.193:60261)
0811 02:52:22.921329 (+ 426us) negotiation.cc:107] Waiting for socket to connect
0811 02:52:22.921344 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 02:52:22.921430 (+ 86us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:52:22.921824 (+ 394us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:52:22.921827 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:52:22.921846 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 02:52:22.922092 (+ 246us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:52:22.922102 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:52:22.923169 (+ 1067us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:52:22.923172 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:52:22.924471 (+ 1299us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:52:22.924478 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:52:22.926372 (+ 1894us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:52:22.926376 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:52:22.926553 (+ 177us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:52:22.926584 (+ 31us) client_negotiation.cc:769] Sending connection context
0811 02:52:22.926690 (+ 106us) client_negotiation.cc:240] Negotiation successful
0811 02:52:22.926769 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":360,"mutex_wait_us":1,"thread_start_us":76,"threads_started":1}
I20260811 02:52:22.927616 21362 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "217c6747ae2c4516817d2f549f1756ed" candidate_uuid: "ab096c76c5544f828e9eb7f2bee1977e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c51f9680930c4419a91d0d0f521006dd" is_pre_election: true
I20260811 02:52:22.927959 21362 raft_consensus.cc:2454] T 217c6747ae2c4516817d2f549f1756ed P c51f9680930c4419a91d0d0f521006dd [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ab096c76c5544f828e9eb7f2bee1977e in term 0.
I20260811 02:52:22.928508 21135 leader_election.cc:302] T 217c6747ae2c4516817d2f549f1756ed P ab096c76c5544f828e9eb7f2bee1977e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ab096c76c5544f828e9eb7f2bee1977e, c51f9680930c4419a91d0d0f521006dd; no voters:
I20260811 02:52:22.928836 21617 raft_consensus.cc:2781] T 217c6747ae2c4516817d2f549f1756ed P ab096c76c5544f828e9eb7f2bee1977e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 02:52:22.928934 21617 raft_consensus.cc:482] T 217c6747ae2c4516817d2f549f1756ed P ab096c76c5544f828e9eb7f2bee1977e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 02:52:22.928993 21617 raft_consensus.cc:3037] T 217c6747ae2c4516817d2f549f1756ed P ab096c76c5544f828e9eb7f2bee1977e [term 0 FOLLOWER]: Advancing to term 1
I20260811 02:52:22.929173 21607 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:52:22.920516 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.138.193:40839 (local address 127.12.138.195:42681)
0811 02:52:22.920635 (+ 119us) server_negotiation.cc:206] Beginning negotiation
0811 02:52:22.920639 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 02:52:22.921222 (+ 583us) server_negotiation.cc:407] Connection header received
0811 02:52:22.921408 (+ 186us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:52:22.921412 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:52:22.921455 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:52:22.921507 (+ 52us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 02:52:22.921980 (+ 473us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:52:22.922510 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:52:22.927868 (+ 5358us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:52:22.928447 (+ 579us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:52:22.928575 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:52:22.928635 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0811 02:52:22.928938 (+ 303us) server_negotiation.cc:299] Negotiation successful
0811 02:52:22.929023 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260811 02:52:22.929581 21621 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:52:22.920207 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.138.195:42681 (local address 127.12.138.193:40839)
0811 02:52:22.921122 (+ 915us) negotiation.cc:107] Waiting for socket to connect
0811 02:52:22.921160 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0811 02:52:22.921266 (+ 106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:52:22.921699 (+ 433us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:52:22.921702 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:52:22.921718 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 02:52:22.921879 (+ 161us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:52:22.921888 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:52:22.922636 (+ 748us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:52:22.922639 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:52:22.924155 (+ 1516us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:52:22.924166 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:52:22.928629 (+ 4463us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:52:22.928633 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:52:22.928749 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:52:22.928787 (+ 38us) client_negotiation.cc:769] Sending connection context
0811 02:52:22.929327 (+ 540us) client_negotiation.cc:240] Negotiation successful
0811 02:52:22.929413 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":839,"mutex_wait_us":26,"thread_start_us":64,"threads_started":1}
I20260811 02:52:22.930635 21617 raft_consensus.cc:504] T 217c6747ae2c4516817d2f549f1756ed P ab096c76c5544f828e9eb7f2bee1977e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ab096c76c5544f828e9eb7f2bee1977e" member_type: VOTER last_known_addr { host: "127.12.138.193" port: 41461 } } peers { permanent_uuid: "adde0ab9117a451c8cf06b46870c7c07" member_type: VOTER last_known_addr { host: "127.12.138.195" port: 42681 } } peers { permanent_uuid: "c51f9680930c4419a91d0d0f521006dd" member_type: VOTER last_known_addr { host: "127.12.138.194" port: 40543 } }
I20260811 02:52:22.931233 21519 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "217c6747ae2c4516817d2f549f1756ed" candidate_uuid: "ab096c76c5544f828e9eb7f2bee1977e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "adde0ab9117a451c8cf06b46870c7c07" is_pre_election: true
I20260811 02:52:22.931516 21362 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "217c6747ae2c4516817d2f549f1756ed" candidate_uuid: "ab096c76c5544f828e9eb7f2bee1977e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c51f9680930c4419a91d0d0f521006dd"
I20260811 02:52:22.931568 21519 raft_consensus.cc:2454] T 217c6747ae2c4516817d2f549f1756ed P adde0ab9117a451c8cf06b46870c7c07 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ab096c76c5544f828e9eb7f2bee1977e in term 0.
I20260811 02:52:22.931659 21362 raft_consensus.cc:3037] T 217c6747ae2c4516817d2f549f1756ed P c51f9680930c4419a91d0d0f521006dd [term 0 FOLLOWER]: Advancing to term 1
I20260811 02:52:22.931778 21518 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "217c6747ae2c4516817d2f549f1756ed" candidate_uuid: "ab096c76c5544f828e9eb7f2bee1977e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "adde0ab9117a451c8cf06b46870c7c07"
I20260811 02:52:22.931885 21518 raft_consensus.cc:3037] T 217c6747ae2c4516817d2f549f1756ed P adde0ab9117a451c8cf06b46870c7c07 [term 0 FOLLOWER]: Advancing to term 1
I20260811 02:52:22.933173 21362 raft_consensus.cc:2454] T 217c6747ae2c4516817d2f549f1756ed P c51f9680930c4419a91d0d0f521006dd [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ab096c76c5544f828e9eb7f2bee1977e in term 1.
I20260811 02:52:22.933283 21518 raft_consensus.cc:2454] T 217c6747ae2c4516817d2f549f1756ed P adde0ab9117a451c8cf06b46870c7c07 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ab096c76c5544f828e9eb7f2bee1977e in term 1.
I20260811 02:52:22.933468 21617 leader_election.cc:288] T 217c6747ae2c4516817d2f549f1756ed P ab096c76c5544f828e9eb7f2bee1977e [CANDIDATE]: Term 1 election: Requested vote from peers adde0ab9117a451c8cf06b46870c7c07 (127.12.138.195:42681), c51f9680930c4419a91d0d0f521006dd (127.12.138.194:40543)
I20260811 02:52:22.933477 21135 leader_election.cc:302] T 217c6747ae2c4516817d2f549f1756ed P ab096c76c5544f828e9eb7f2bee1977e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ab096c76c5544f828e9eb7f2bee1977e, c51f9680930c4419a91d0d0f521006dd; no voters:
I20260811 02:52:22.933640 21617 raft_consensus.cc:2781] T 217c6747ae2c4516817d2f549f1756ed P ab096c76c5544f828e9eb7f2bee1977e [term 1 FOLLOWER]: Leader election won for term 1
I20260811 02:52:22.933866 21617 raft_consensus.cc:686] T 217c6747ae2c4516817d2f549f1756ed P ab096c76c5544f828e9eb7f2bee1977e [term 1 LEADER]: Becoming Leader. State: Replica: ab096c76c5544f828e9eb7f2bee1977e, State: Running, Role: LEADER
I20260811 02:52:22.934203 21617 consensus_queue.cc:237] T 217c6747ae2c4516817d2f549f1756ed P ab096c76c5544f828e9eb7f2bee1977e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ab096c76c5544f828e9eb7f2bee1977e" member_type: VOTER last_known_addr { host: "127.12.138.193" port: 41461 } } peers { permanent_uuid: "adde0ab9117a451c8cf06b46870c7c07" member_type: VOTER last_known_addr { host: "127.12.138.195" port: 42681 } } peers { permanent_uuid: "c51f9680930c4419a91d0d0f521006dd" member_type: VOTER last_known_addr { host: "127.12.138.194" port: 40543 } }
I20260811 02:52:22.938246 20136 catalog_manager.cc:5495] T 217c6747ae2c4516817d2f549f1756ed P ab096c76c5544f828e9eb7f2bee1977e reported cstate change: term changed from 0 to 1, leader changed from <none> to ab096c76c5544f828e9eb7f2bee1977e (127.12.138.193). New cstate: current_term: 1 leader_uuid: "ab096c76c5544f828e9eb7f2bee1977e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ab096c76c5544f828e9eb7f2bee1977e" member_type: VOTER last_known_addr { host: "127.12.138.193" port: 41461 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "adde0ab9117a451c8cf06b46870c7c07" member_type: VOTER last_known_addr { host: "127.12.138.195" port: 42681 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c51f9680930c4419a91d0d0f521006dd" member_type: VOTER last_known_addr { host: "127.12.138.194" port: 40543 } health_report { overall_health: UNKNOWN } } }
I20260811 02:52:22.970814 21605 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:52:22.967447 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34060 (local address 127.12.138.193:41461)
0811 02:52:22.967600 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0811 02:52:22.967604 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 02:52:22.967617 (+ 13us) server_negotiation.cc:407] Connection header received
0811 02:52:22.967696 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:52:22.967698 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:52:22.967752 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:52:22.967861 (+ 109us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 02:52:22.968083 (+ 222us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:52:22.968585 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:52:22.969495 (+ 910us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:52:22.969703 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:52:22.969791 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:52:22.970043 (+ 252us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 02:52:22.970466 (+ 423us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 02:52:22.970558 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0811 02:52:22.970633 (+ 75us) server_negotiation.cc:299] Negotiation successful
0811 02:52:22.970684 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260811 02:52:22.976738 21518 raft_consensus.cc:1261] T 217c6747ae2c4516817d2f549f1756ed P adde0ab9117a451c8cf06b46870c7c07 [term 1 FOLLOWER]: Refusing update from remote peer ab096c76c5544f828e9eb7f2bee1977e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 02:52:22.976766 21362 raft_consensus.cc:1261] T 217c6747ae2c4516817d2f549f1756ed P c51f9680930c4419a91d0d0f521006dd [term 1 FOLLOWER]: Refusing update from remote peer ab096c76c5544f828e9eb7f2bee1977e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 02:52:22.977589 21622 consensus_queue.cc:1035] T 217c6747ae2c4516817d2f549f1756ed P ab096c76c5544f828e9eb7f2bee1977e [LEADER]: Connected to new peer: Peer: permanent_uuid: "c51f9680930c4419a91d0d0f521006dd" member_type: VOTER last_known_addr { host: "127.12.138.194" port: 40543 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 02:52:22.977841 21617 consensus_queue.cc:1035] T 217c6747ae2c4516817d2f549f1756ed P ab096c76c5544f828e9eb7f2bee1977e [LEADER]: Connected to new peer: Peer: permanent_uuid: "adde0ab9117a451c8cf06b46870c7c07" member_type: VOTER last_known_addr { host: "127.12.138.195" port: 42681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 02:52:22.985685 21625 mvcc.cc:205] Tried to move back new op lower bound from 7317162979225337856 to 7317162979061420032. Current Snapshot: MvccSnapshot[applied={T|T < 7317162979225337856}]
I20260811 02:52:22.986296 21626 mvcc.cc:205] Tried to move back new op lower bound from 7317162979225337856 to 7317162979061420032. Current Snapshot: MvccSnapshot[applied={T|T < 7317162979225337856}]
I20260811 02:52:22.990795 21627 mvcc.cc:205] Tried to move back new op lower bound from 7317162979225337856 to 7317162979061420032. Current Snapshot: MvccSnapshot[applied={T|T < 7317162979225337856}]
I20260811 02:52:22.995507 20136 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39466:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 02:52:22.995674 20136 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39466:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 02:52:23.000383 20136 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0317e693c61849e9bf42d55fbf6ef321: Sending DeleteTablet for 3 replicas of tablet 217c6747ae2c4516817d2f549f1756ed
I20260811 02:52:23.001288 21185 tablet_service.cc:1514] Processing DeleteTablet for tablet 217c6747ae2c4516817d2f549f1756ed with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 02:52:22 UTC) from {username='oryx', principal='oryx/127.12.138.254@KRBTEST.COM'} at 127.0.0.1:34046
I20260811 02:52:23.001971 21636 tablet_replica.cc:330] T 217c6747ae2c4516817d2f549f1756ed P ab096c76c5544f828e9eb7f2bee1977e: stopping tablet replica
I20260811 02:52:23.002087 21499 tablet_service.cc:1514] Processing DeleteTablet for tablet 217c6747ae2c4516817d2f549f1756ed with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 02:52:22 UTC) from {username='oryx', principal='oryx/127.12.138.254@KRBTEST.COM'} at 127.0.0.1:36444
I20260811 02:52:23.002547 21636 raft_consensus.cc:2229] T 217c6747ae2c4516817d2f549f1756ed P ab096c76c5544f828e9eb7f2bee1977e [term 1 LEADER]: Raft consensus shutting down.
I20260811 02:52:23.002955 21636 raft_consensus.cc:2258] T 217c6747ae2c4516817d2f549f1756ed P ab096c76c5544f828e9eb7f2bee1977e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 02:52:23.003629 21340 tablet_service.cc:1514] Processing DeleteTablet for tablet 217c6747ae2c4516817d2f549f1756ed with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 02:52:22 UTC) from {username='oryx', principal='oryx/127.12.138.254@KRBTEST.COM'} at 127.0.0.1:52766
I20260811 02:52:23.003880 21638 tablet_replica.cc:330] T 217c6747ae2c4516817d2f549f1756ed P adde0ab9117a451c8cf06b46870c7c07: stopping tablet replica
I20260811 02:52:23.004195 21638 raft_consensus.cc:2229] T 217c6747ae2c4516817d2f549f1756ed P adde0ab9117a451c8cf06b46870c7c07 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 02:52:23.004583 21638 raft_consensus.cc:2258] T 217c6747ae2c4516817d2f549f1756ed P adde0ab9117a451c8cf06b46870c7c07 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 02:52:23.005043 21636 ts_tablet_manager.cc:1905] T 217c6747ae2c4516817d2f549f1756ed P ab096c76c5544f828e9eb7f2bee1977e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 02:52:23.005302 12843 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDawEwS/build/debug/bin/kudu with pid 20094
I20260811 02:52:23.006285 21638 ts_tablet_manager.cc:1905] T 217c6747ae2c4516817d2f549f1756ed P adde0ab9117a451c8cf06b46870c7c07: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 02:52:23.008524 21636 ts_tablet_manager.cc:1918] T 217c6747ae2c4516817d2f549f1756ed P ab096c76c5544f828e9eb7f2bee1977e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 02:52:23.008710 21636 log.cc:1196] T 217c6747ae2c4516817d2f549f1756ed P ab096c76c5544f828e9eb7f2bee1977e: Deleting WAL directory at /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786416707705494-12843-0/minicluster-data/ts-0/wal/wals/217c6747ae2c4516817d2f549f1756ed
I20260811 02:52:23.009181 21636 ts_tablet_manager.cc:1939] T 217c6747ae2c4516817d2f549f1756ed P ab096c76c5544f828e9eb7f2bee1977e: Deleting consensus metadata
I20260811 02:52:23.009406 21640 tablet_replica.cc:330] T 217c6747ae2c4516817d2f549f1756ed P c51f9680930c4419a91d0d0f521006dd: stopping tablet replica
I20260811 02:52:23.009783 21640 raft_consensus.cc:2229] T 217c6747ae2c4516817d2f549f1756ed P c51f9680930c4419a91d0d0f521006dd [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 02:52:23.010140 21640 raft_consensus.cc:2258] T 217c6747ae2c4516817d2f549f1756ed P c51f9680930c4419a91d0d0f521006dd [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 02:52:23.011655 21640 ts_tablet_manager.cc:1905] T 217c6747ae2c4516817d2f549f1756ed P c51f9680930c4419a91d0d0f521006dd: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 02:52:23.013937 21640 ts_tablet_manager.cc:1918] T 217c6747ae2c4516817d2f549f1756ed P c51f9680930c4419a91d0d0f521006dd: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 02:52:23.014068 21640 log.cc:1196] T 217c6747ae2c4516817d2f549f1756ed P c51f9680930c4419a91d0d0f521006dd: Deleting WAL directory at /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786416707705494-12843-0/minicluster-data/ts-1/wal/wals/217c6747ae2c4516817d2f549f1756ed
I20260811 02:52:23.014372 21640 ts_tablet_manager.cc:1939] T 217c6747ae2c4516817d2f549f1756ed P c51f9680930c4419a91d0d0f521006dd: Deleting consensus metadata
I20260811 02:52:23.015898 21638 ts_tablet_manager.cc:1918] T 217c6747ae2c4516817d2f549f1756ed P adde0ab9117a451c8cf06b46870c7c07: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 02:52:23.016033 21638 log.cc:1196] T 217c6747ae2c4516817d2f549f1756ed P adde0ab9117a451c8cf06b46870c7c07: Deleting WAL directory at /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786416707705494-12843-0/minicluster-data/ts-2/wal/wals/217c6747ae2c4516817d2f549f1756ed
I20260811 02:52:23.016374 21638 ts_tablet_manager.cc:1939] T 217c6747ae2c4516817d2f549f1756ed P adde0ab9117a451c8cf06b46870c7c07: Deleting consensus metadata
W20260811 02:52:23.020509 21289 connection.cc:535] server connection from 127.0.0.1:52766 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260811 02:52:23.020530 21448 connection.cc:535] server connection from 127.0.0.1:36444 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260811 02:52:23.020996 21134 connection.cc:535] server connection from 127.0.0.1:34046 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260811 02:52:23.031311 21590 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:52:23.004156 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44236 (local address 127.12.138.253:42467)
0811 02:52:23.004341 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0811 02:52:23.004347 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 02:52:23.004383 (+ 36us) server_negotiation.cc:407] Connection header received
0811 02:52:23.004425 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:52:23.004428 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:52:23.004478 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:52:23.004563 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:52:23.022045 (+ 17482us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:52:23.022737 (+ 692us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:52:23.024017 (+ 1280us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:52:23.024337 (+ 320us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:52:23.024458 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:52:23.027105 (+ 2647us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:52:23.027109 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:52:23.027112 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:52:23.027142 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:52:23.029066 (+ 1924us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:52:23.029611 (+ 545us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:52:23.029615 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:52:23.029617 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:52:23.029708 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:52:23.030299 (+ 591us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:52:23.030303 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:52:23.030305 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:52:23.030523 (+ 218us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:52:23.030639 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0811 02:52:23.030993 (+ 354us) server_negotiation.cc:299] Negotiation successful
0811 02:52:23.031129 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260811 02:52:23.035238 20977 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:52:23.025308 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44246 (local address 127.12.138.253:42467)
0811 02:52:23.025435 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0811 02:52:23.025442 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 02:52:23.025456 (+ 14us) server_negotiation.cc:407] Connection header received
0811 02:52:23.025611 (+ 155us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:52:23.025615 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:52:23.025668 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:52:23.025722 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:52:23.026984 (+ 1262us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:52:23.027777 (+ 793us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:52:23.029013 (+ 1236us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:52:23.029430 (+ 417us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:52:23.029509 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:52:23.031206 (+ 1697us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:52:23.031209 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:52:23.031212 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:52:23.031239 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:52:23.033104 (+ 1865us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:52:23.033673 (+ 569us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:52:23.033678 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:52:23.033682 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:52:23.033776 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:52:23.034127 (+ 351us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:52:23.034135 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:52:23.034138 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:52:23.034403 (+ 265us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:52:23.034525 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0811 02:52:23.034929 (+ 404us) server_negotiation.cc:299] Negotiation successful
0811 02:52:23.035054 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260811 02:52:23.037880 21591 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:52:23.025723 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34414 (local address 127.12.138.252:42917)
0811 02:52:23.025884 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0811 02:52:23.025891 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 02:52:23.026309 (+ 418us) server_negotiation.cc:407] Connection header received
0811 02:52:23.027136 (+ 827us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:52:23.027140 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:52:23.027193 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:52:23.027246 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:52:23.028201 (+ 955us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:52:23.029037 (+ 836us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:52:23.030381 (+ 1344us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:52:23.030841 (+ 460us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:52:23.030929 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:52:23.033131 (+ 2202us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:52:23.033190 (+ 59us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:52:23.033197 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:52:23.033228 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:52:23.035112 (+ 1884us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:52:23.035940 (+ 828us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:52:23.035944 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:52:23.035945 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:52:23.036045 (+ 100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:52:23.036374 (+ 329us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:52:23.036380 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:52:23.036383 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:52:23.036594 (+ 211us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:52:23.036697 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0811 02:52:23.037116 (+ 419us) server_negotiation.cc:299] Negotiation successful
0811 02:52:23.037753 (+ 637us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":420,"server-negotiator.queue_time_us":44}
I20260811 02:52:23.045346 21266 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:52:23.004725 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34408 (local address 127.12.138.252:42917)
0811 02:52:23.004860 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0811 02:52:23.004868 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 02:52:23.033227 (+ 28359us) server_negotiation.cc:407] Connection header received
0811 02:52:23.033413 (+ 186us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:52:23.033417 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:52:23.033462 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:52:23.033511 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:52:23.036931 (+ 3420us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:52:23.038415 (+ 1484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:52:23.039556 (+ 1141us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:52:23.039770 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:52:23.039903 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:52:23.041685 (+ 1782us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:52:23.041689 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:52:23.041692 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:52:23.041714 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:52:23.043586 (+ 1872us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:52:23.044023 (+ 437us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:52:23.044027 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:52:23.044029 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:52:23.044116 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:52:23.044405 (+ 289us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:52:23.044410 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:52:23.044411 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:52:23.044605 (+ 194us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:52:23.044749 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0811 02:52:23.044990 (+ 241us) server_negotiation.cc:299] Negotiation successful
0811 02:52:23.045112 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260811 02:52:23.991113 21648 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:52:23.990675 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.138.254:43457 (local address 127.12.138.195:38691)
0811 02:52:23.990934 (+ 259us) negotiation.cc:107] Waiting for socket to connect
0811 02:52:23.991017 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.138.254:43457: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":187,"thread_start_us":96,"threads_started":1}
W20260811 02:52:23.991456 21574 heartbeater.cc:644] Failed to heartbeat to 127.12.138.254:43457 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.12.138.254:43457: connect: Connection refused (error 111)
I20260811 02:52:23.993424 21649 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:52:23.993003 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.138.254:43457 (local address 127.12.138.193:39669)
0811 02:52:23.993252 (+ 249us) negotiation.cc:107] Waiting for socket to connect
0811 02:52:23.993342 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.138.254:43457: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":176,"thread_start_us":80,"threads_started":1}
W20260811 02:52:23.993695 21261 heartbeater.cc:644] Failed to heartbeat to 127.12.138.254:43457 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.12.138.254:43457: connect: Connection refused (error 111)
I20260811 02:52:23.995061 21650 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:52:23.994572 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.138.254:43457 (local address 127.12.138.194:38325)
0811 02:52:23.994862 (+ 290us) negotiation.cc:107] Waiting for socket to connect
0811 02:52:23.994959 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.138.254:43457: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":190,"thread_start_us":71,"threads_started":1}
W20260811 02:52:23.995352 21419 heartbeater.cc:644] Failed to heartbeat to 127.12.138.254:43457 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.12.138.254:43457: connect: Connection refused (error 111)
I20260811 02:52:24.500838 21651 raft_consensus.cc:482] T 00000000000000000000000000000000 P 269d7f529b994b32bc6d6d1b512371d4 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 0317e693c61849e9bf42d55fbf6ef321)
I20260811 02:52:24.500828 21652 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3b8a3f24429b46a29f1697d97b2da6d1 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 0317e693c61849e9bf42d55fbf6ef321)
I20260811 02:52:24.501008 21651 raft_consensus.cc:504] T 00000000000000000000000000000000 P 269d7f529b994b32bc6d6d1b512371d4 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "0317e693c61849e9bf42d55fbf6ef321" member_type: VOTER last_known_addr { host: "127.12.138.254" port: 43457 } } peers { permanent_uuid: "269d7f529b994b32bc6d6d1b512371d4" member_type: VOTER last_known_addr { host: "127.12.138.253" port: 42467 } attrs { promote: false } } peers { permanent_uuid: "3b8a3f24429b46a29f1697d97b2da6d1" member_type: VOTER last_known_addr { host: "127.12.138.252" port: 42917 } attrs { promote: false } }
I20260811 02:52:24.501008 21652 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3b8a3f24429b46a29f1697d97b2da6d1 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "0317e693c61849e9bf42d55fbf6ef321" member_type: VOTER last_known_addr { host: "127.12.138.254" port: 43457 } } peers { permanent_uuid: "269d7f529b994b32bc6d6d1b512371d4" member_type: VOTER last_known_addr { host: "127.12.138.253" port: 42467 } attrs { promote: false } } peers { permanent_uuid: "3b8a3f24429b46a29f1697d97b2da6d1" member_type: VOTER last_known_addr { host: "127.12.138.252" port: 42917 } attrs { promote: false } }
I20260811 02:52:24.502015 21651 leader_election.cc:288] T 00000000000000000000000000000000 P 269d7f529b994b32bc6d6d1b512371d4 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0317e693c61849e9bf42d55fbf6ef321 (127.12.138.254:43457), 3b8a3f24429b46a29f1697d97b2da6d1 (127.12.138.252:42917)
I20260811 02:52:24.502105 21652 leader_election.cc:288] T 00000000000000000000000000000000 P 3b8a3f24429b46a29f1697d97b2da6d1 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0317e693c61849e9bf42d55fbf6ef321 (127.12.138.254:43457), 269d7f529b994b32bc6d6d1b512371d4 (127.12.138.253:42467)
I20260811 02:52:24.502408 21654 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:52:24.501699 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.138.254:43457 (local address 127.0.0.1:39958)
0811 02:52:24.502204 (+ 505us) negotiation.cc:107] Waiting for socket to connect
0811 02:52:24.502297 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.138.254:43457: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":442,"thread_start_us":75,"threads_started":1}
W20260811 02:52:24.502591 20821 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.12.138.254:43457: connect: Connection refused (error 111)
I20260811 02:52:24.503214 21653 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:52:24.501699 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.138.254:43457 (local address 127.0.0.1:39964)
0811 02:52:24.503028 (+ 1329us) negotiation.cc:107] Waiting for socket to connect
0811 02:52:24.503123 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.138.254:43457: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":290,"thread_start_us":63,"threads_started":1}
W20260811 02:52:24.503504 21037 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.12.138.254:43457: connect: Connection refused (error 111)
I20260811 02:52:24.503639 21654 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:52:24.503373 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.138.254:43457 (local address 127.0.0.1:39970)
0811 02:52:24.503507 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0811 02:52:24.503565 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.138.254:43457: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":72}
W20260811 02:52:24.503749 20821 leader_election.cc:334] T 00000000000000000000000000000000 P 269d7f529b994b32bc6d6d1b512371d4 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 0317e693c61849e9bf42d55fbf6ef321 (127.12.138.254:43457): Network error: Client connection negotiation failed: client connection to 127.12.138.254:43457: connect: Connection refused (error 111)
I20260811 02:52:24.504587 21653 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:52:24.504352 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.138.254:43457 (local address 127.0.0.1:39980)
0811 02:52:24.504478 (+ 126us) negotiation.cc:107] Waiting for socket to connect
0811 02:52:24.504528 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.138.254:43457: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
W20260811 02:52:24.504729 21037 leader_election.cc:334] T 00000000000000000000000000000000 P 3b8a3f24429b46a29f1697d97b2da6d1 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 0317e693c61849e9bf42d55fbf6ef321 (127.12.138.254:43457): Network error: Client connection negotiation failed: client connection to 127.12.138.254:43457: connect: Connection refused (error 111)
I20260811 02:52:24.506887 21657 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:52:24.502520 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.138.252:42917 (local address 127.0.0.1:34418)
0811 02:52:24.502762 (+ 242us) negotiation.cc:107] Waiting for socket to connect
0811 02:52:24.502776 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 02:52:24.502876 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:52:24.503398 (+ 522us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:52:24.503402 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:52:24.503441 (+ 39us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 02:52:24.503562 (+ 121us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:52:24.503572 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:52:24.504801 (+ 1229us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:52:24.504804 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:52:24.505750 (+ 946us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:52:24.505759 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:52:24.506438 (+ 679us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:52:24.506442 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:52:24.506532 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:52:24.506572 (+ 40us) client_negotiation.cc:769] Sending connection context
0811 02:52:24.506647 (+ 75us) client_negotiation.cc:240] Negotiation successful
0811 02:52:24.506719 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":188,"spinlock_wait_cycles":4480,"thread_start_us":68,"threads_started":1}
I20260811 02:52:24.506898 21659 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:52:24.502855 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34418 (local address 127.12.138.252:42917)
0811 02:52:24.503140 (+ 285us) server_negotiation.cc:206] Beginning negotiation
0811 02:52:24.503145 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 02:52:24.503158 (+ 13us) server_negotiation.cc:407] Connection header received
0811 02:52:24.503205 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:52:24.503208 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:52:24.503258 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:52:24.503692 (+ 434us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 02:52:24.503936 (+ 244us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:52:24.504702 (+ 766us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:52:24.505875 (+ 1173us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:52:24.506315 (+ 440us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:52:24.506406 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:52:24.506447 (+ 41us) server_negotiation.cc:1035] Waiting for connection context
0811 02:52:24.506671 (+ 224us) server_negotiation.cc:299] Negotiation successful
0811 02:52:24.506744 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":66,"server-negotiator.queue_time_us":222,"thread_start_us":52,"threads_started":1}
I20260811 02:52:24.507078 21656 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:52:24.502162 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44260 (local address 127.12.138.253:42467)
0811 02:52:24.503335 (+ 1173us) server_negotiation.cc:206] Beginning negotiation
0811 02:52:24.503340 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 02:52:24.503352 (+ 12us) server_negotiation.cc:407] Connection header received
0811 02:52:24.503394 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:52:24.503397 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:52:24.503464 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:52:24.503913 (+ 449us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 02:52:24.504019 (+ 106us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:52:24.504789 (+ 770us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:52:24.506023 (+ 1234us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:52:24.506492 (+ 469us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:52:24.506695 (+ 203us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:52:24.506742 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0811 02:52:24.506857 (+ 115us) server_negotiation.cc:299] Negotiation successful
0811 02:52:24.506933 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1102,"spinlock_wait_cycles":14592,"thread_start_us":103,"threads_started":1}
I20260811 02:52:24.507336 21069 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "269d7f529b994b32bc6d6d1b512371d4" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "3b8a3f24429b46a29f1697d97b2da6d1" is_pre_election: true
I20260811 02:52:24.507485 21655 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:52:24.501782 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.138.253:42467 (local address 127.0.0.1:44260)
0811 02:52:24.503047 (+ 1265us) negotiation.cc:107] Waiting for socket to connect
0811 02:52:24.503062 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 02:52:24.503152 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:52:24.503591 (+ 439us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:52:24.503594 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:52:24.503621 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 02:52:24.503789 (+ 168us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:52:24.503808 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:52:24.504910 (+ 1102us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:52:24.504914 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:52:24.505865 (+ 951us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:52:24.505874 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:52:24.506635 (+ 761us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:52:24.506641 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:52:24.506726 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:52:24.506747 (+ 21us) client_negotiation.cc:769] Sending connection context
0811 02:52:24.506831 (+ 84us) client_negotiation.cc:240] Negotiation successful
0811 02:52:24.506893 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":561,"mutex_wait_us":3,"thread_start_us":62,"threads_started":1}
I20260811 02:52:24.507563 21069 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 3b8a3f24429b46a29f1697d97b2da6d1 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 269d7f529b994b32bc6d6d1b512371d4 in term 1.
I20260811 02:52:24.507863 20824 leader_election.cc:302] T 00000000000000000000000000000000 P 269d7f529b994b32bc6d6d1b512371d4 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 269d7f529b994b32bc6d6d1b512371d4, 3b8a3f24429b46a29f1697d97b2da6d1; no voters: 0317e693c61849e9bf42d55fbf6ef321
I20260811 02:52:24.508062 21651 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 269d7f529b994b32bc6d6d1b512371d4 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260811 02:52:24.508159 21651 raft_consensus.cc:482] T 00000000000000000000000000000000 P 269d7f529b994b32bc6d6d1b512371d4 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 0317e693c61849e9bf42d55fbf6ef321)
I20260811 02:52:24.508209 21651 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 269d7f529b994b32bc6d6d1b512371d4 [term 1 FOLLOWER]: Advancing to term 2
I20260811 02:52:24.508333 20849 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "3b8a3f24429b46a29f1697d97b2da6d1" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "269d7f529b994b32bc6d6d1b512371d4" is_pre_election: true
I20260811 02:52:24.509451 21651 raft_consensus.cc:504] T 00000000000000000000000000000000 P 269d7f529b994b32bc6d6d1b512371d4 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "0317e693c61849e9bf42d55fbf6ef321" member_type: VOTER last_known_addr { host: "127.12.138.254" port: 43457 } } peers { permanent_uuid: "269d7f529b994b32bc6d6d1b512371d4" member_type: VOTER last_known_addr { host: "127.12.138.253" port: 42467 } attrs { promote: false } } peers { permanent_uuid: "3b8a3f24429b46a29f1697d97b2da6d1" member_type: VOTER last_known_addr { host: "127.12.138.252" port: 42917 } attrs { promote: false } }
I20260811 02:52:24.509696 20849 raft_consensus.cc:2379] T 00000000000000000000000000000000 P 269d7f529b994b32bc6d6d1b512371d4 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3b8a3f24429b46a29f1697d97b2da6d1 in current term 2: Already voted for candidate 269d7f529b994b32bc6d6d1b512371d4 in this term.
I20260811 02:52:24.509847 21651 leader_election.cc:288] T 00000000000000000000000000000000 P 269d7f529b994b32bc6d6d1b512371d4 [CANDIDATE]: Term 2 election: Requested vote from peers 0317e693c61849e9bf42d55fbf6ef321 (127.12.138.254:43457), 3b8a3f24429b46a29f1697d97b2da6d1 (127.12.138.252:42917)
I20260811 02:52:24.510129 21657 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:52:24.509926 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.138.254:43457 (local address 127.0.0.1:39984)
0811 02:52:24.510006 (+ 80us) negotiation.cc:107] Waiting for socket to connect
0811 02:52:24.510057 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.138.254:43457: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":23}
I20260811 02:52:24.510353 21069 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "269d7f529b994b32bc6d6d1b512371d4" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "3b8a3f24429b46a29f1697d97b2da6d1"
I20260811 02:52:24.510357 21038 leader_election.cc:302] T 00000000000000000000000000000000 P 3b8a3f24429b46a29f1697d97b2da6d1 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b8a3f24429b46a29f1697d97b2da6d1; no voters: 0317e693c61849e9bf42d55fbf6ef321, 269d7f529b994b32bc6d6d1b512371d4
I20260811 02:52:24.510488 21069 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3b8a3f24429b46a29f1697d97b2da6d1 [term 1 FOLLOWER]: Advancing to term 2
I20260811 02:52:24.510840 21657 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:52:24.510583 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.138.254:43457 (local address 127.0.0.1:40000)
0811 02:52:24.510756 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0811 02:52:24.510796 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.138.254:43457: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":113}
W20260811 02:52:24.510977 20821 leader_election.cc:334] T 00000000000000000000000000000000 P 269d7f529b994b32bc6d6d1b512371d4 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 0317e693c61849e9bf42d55fbf6ef321 (127.12.138.254:43457): Network error: Client connection negotiation failed: client connection to 127.12.138.254:43457: connect: Connection refused (error 111)
I20260811 02:52:24.511613 21069 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 3b8a3f24429b46a29f1697d97b2da6d1 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 269d7f529b994b32bc6d6d1b512371d4 in term 2.
I20260811 02:52:24.511761 21652 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 3b8a3f24429b46a29f1697d97b2da6d1 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260811 02:52:24.511885 20824 leader_election.cc:302] T 00000000000000000000000000000000 P 269d7f529b994b32bc6d6d1b512371d4 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 269d7f529b994b32bc6d6d1b512371d4, 3b8a3f24429b46a29f1697d97b2da6d1; no voters: 0317e693c61849e9bf42d55fbf6ef321
I20260811 02:52:24.512082 21651 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 269d7f529b994b32bc6d6d1b512371d4 [term 2 FOLLOWER]: Leader election won for term 2
I20260811 02:52:24.512341 21651 raft_consensus.cc:686] T 00000000000000000000000000000000 P 269d7f529b994b32bc6d6d1b512371d4 [term 2 LEADER]: Becoming Leader. State: Replica: 269d7f529b994b32bc6d6d1b512371d4, State: Running, Role: LEADER
I20260811 02:52:24.512606 21651 consensus_queue.cc:237] T 00000000000000000000000000000000 P 269d7f529b994b32bc6d6d1b512371d4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 1.12, Last appended by leader: 12, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "0317e693c61849e9bf42d55fbf6ef321" member_type: VOTER last_known_addr { host: "127.12.138.254" port: 43457 } } peers { permanent_uuid: "269d7f529b994b32bc6d6d1b512371d4" member_type: VOTER last_known_addr { host: "127.12.138.253" port: 42467 } attrs { promote: false } } peers { permanent_uuid: "3b8a3f24429b46a29f1697d97b2da6d1" member_type: VOTER last_known_addr { host: "127.12.138.252" port: 42917 } attrs { promote: false } }
I20260811 02:52:24.513767 21661 sys_catalog.cc:455] T 00000000000000000000000000000000 P 269d7f529b994b32bc6d6d1b512371d4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 269d7f529b994b32bc6d6d1b512371d4. Latest consensus state: current_term: 2 leader_uuid: "269d7f529b994b32bc6d6d1b512371d4" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "0317e693c61849e9bf42d55fbf6ef321" member_type: VOTER last_known_addr { host: "127.12.138.254" port: 43457 } } peers { permanent_uuid: "269d7f529b994b32bc6d6d1b512371d4" member_type: VOTER last_known_addr { host: "127.12.138.253" port: 42467 } attrs { promote: false } } peers { permanent_uuid: "3b8a3f24429b46a29f1697d97b2da6d1" member_type: VOTER last_known_addr { host: "127.12.138.252" port: 42917 } attrs { promote: false } } }
I20260811 02:52:24.513862 21661 sys_catalog.cc:458] T 00000000000000000000000000000000 P 269d7f529b994b32bc6d6d1b512371d4 [sys.catalog]: This master's current role is: LEADER
I20260811 02:52:24.514149 21663 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 02:52:24.516278 21663 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 02:52:24.516690 21663 catalog_manager.cc:1259] Loaded cluster ID: 101bfd3e0cf345c7b289e94b98d49ad6
I20260811 02:52:24.516736 21663 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 02:52:24.517369 21663 catalog_manager.cc:1504] Loading token signing keys...
I20260811 02:52:24.517767 21663 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 269d7f529b994b32bc6d6d1b512371d4: Loaded TSK: 0
I20260811 02:52:24.518451 21663 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 02:52:24.566785 20830 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44246:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 02:52:24.569981 20830 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 02:52:24.573391 21657 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:52:24.573101 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.138.254:43457 (local address 127.0.0.1:40024)
0811 02:52:24.573251 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0811 02:52:24.573321 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.138.254:43457: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":72}
I20260811 02:52:24.573559 21069 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 3b8a3f24429b46a29f1697d97b2da6d1 [term 2 FOLLOWER]: Refusing update from remote peer 269d7f529b994b32bc6d6d1b512371d4: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 2 index: 14. (index mismatch)
I20260811 02:52:24.574218 21657 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:52:24.573974 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.138.254:43457 (local address 127.0.0.1:40030)
0811 02:52:24.574062 (+ 88us) negotiation.cc:107] Waiting for socket to connect
0811 02:52:24.574111 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.138.254:43457: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":27}
I20260811 02:52:24.574129 21661 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 269d7f529b994b32bc6d6d1b512371d4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b8a3f24429b46a29f1697d97b2da6d1" member_type: VOTER last_known_addr { host: "127.12.138.252" port: 42917 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
W20260811 02:52:24.574457 20821 consensus_peers.cc:487] T 00000000000000000000000000000000 P 269d7f529b994b32bc6d6d1b512371d4 -> Peer 0317e693c61849e9bf42d55fbf6ef321 (127.12.138.254:43457): Couldn't send request to peer 0317e693c61849e9bf42d55fbf6ef321. Status: Network error: Client connection negotiation failed: client connection to 127.12.138.254:43457: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260811 02:52:24.575906 21652 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3b8a3f24429b46a29f1697d97b2da6d1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 269d7f529b994b32bc6d6d1b512371d4. Latest consensus state: current_term: 2 leader_uuid: "269d7f529b994b32bc6d6d1b512371d4" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "0317e693c61849e9bf42d55fbf6ef321" member_type: VOTER last_known_addr { host: "127.12.138.254" port: 43457 } } peers { permanent_uuid: "269d7f529b994b32bc6d6d1b512371d4" member_type: VOTER last_known_addr { host: "127.12.138.253" port: 42467 } attrs { promote: false } } peers { permanent_uuid: "3b8a3f24429b46a29f1697d97b2da6d1" member_type: VOTER last_known_addr { host: "127.12.138.252" port: 42917 } attrs { promote: false } } }
I20260811 02:52:24.576063 21652 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3b8a3f24429b46a29f1697d97b2da6d1 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 02:52:24.577193 21661 sys_catalog.cc:455] T 00000000000000000000000000000000 P 269d7f529b994b32bc6d6d1b512371d4 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 2 leader_uuid: "269d7f529b994b32bc6d6d1b512371d4" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "0317e693c61849e9bf42d55fbf6ef321" member_type: VOTER last_known_addr { host: "127.12.138.254" port: 43457 } } peers { permanent_uuid: "269d7f529b994b32bc6d6d1b512371d4" member_type: VOTER last_known_addr { host: "127.12.138.253" port: 42467 } attrs { promote: false } } peers { permanent_uuid: "3b8a3f24429b46a29f1697d97b2da6d1" member_type: VOTER last_known_addr { host: "127.12.138.252" port: 42917 } attrs { promote: false } } }
I20260811 02:52:24.577316 21661 sys_catalog.cc:458] T 00000000000000000000000000000000 P 269d7f529b994b32bc6d6d1b512371d4 [sys.catalog]: This master's current role is: LEADER
I20260811 02:52:24.577488 21652 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3b8a3f24429b46a29f1697d97b2da6d1 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 2 leader_uuid: "269d7f529b994b32bc6d6d1b512371d4" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "0317e693c61849e9bf42d55fbf6ef321" member_type: VOTER last_known_addr { host: "127.12.138.254" port: 43457 } } peers { permanent_uuid: "269d7f529b994b32bc6d6d1b512371d4" member_type: VOTER last_known_addr { host: "127.12.138.253" port: 42467 } attrs { promote: false } } peers { permanent_uuid: "3b8a3f24429b46a29f1697d97b2da6d1" member_type: VOTER last_known_addr { host: "127.12.138.252" port: 42917 } attrs { promote: false } } }
I20260811 02:52:24.577611 21652 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3b8a3f24429b46a29f1697d97b2da6d1 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 02:52:24.589439 21673 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:52:24.584691 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.138.194:40543 (local address 127.0.0.1:52778)
0811 02:52:24.584992 (+ 301us) negotiation.cc:107] Waiting for socket to connect
0811 02:52:24.585006 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 02:52:24.585096 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:52:24.585497 (+ 401us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:52:24.585500 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:52:24.585514 (+ 14us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 02:52:24.585599 (+ 85us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:52:24.585603 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:52:24.586805 (+ 1202us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:52:24.586809 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:52:24.588158 (+ 1349us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:52:24.588170 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:52:24.588948 (+ 778us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:52:24.588952 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:52:24.589046 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:52:24.589070 (+ 24us) client_negotiation.cc:769] Sending connection context
0811 02:52:24.589206 (+ 136us) client_negotiation.cc:240] Negotiation successful
0811 02:52:24.589296 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":232,"thread_start_us":114,"threads_started":1}
I20260811 02:52:24.589452 21674 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:52:24.584817 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52778 (local address 127.12.138.194:40543)
0811 02:52:24.585104 (+ 287us) server_negotiation.cc:206] Beginning negotiation
0811 02:52:24.585109 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 02:52:24.585124 (+ 15us) server_negotiation.cc:407] Connection header received
0811 02:52:24.585212 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:52:24.585215 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:52:24.585261 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:52:24.585370 (+ 109us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 02:52:24.585714 (+ 344us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:52:24.586642 (+ 928us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:52:24.588294 (+ 1652us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:52:24.588696 (+ 402us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:52:24.588963 (+ 267us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:52:24.589019 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0811 02:52:24.589257 (+ 238us) server_negotiation.cc:299] Negotiation successful
0811 02:52:24.589342 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":216,"thread_start_us":99,"threads_started":1}
I20260811 02:52:24.590596 21340 tablet_service.cc:1467] Processing CreateTablet for tablet ea888b104dd0418f974ddcb758c07237 (DEFAULT_TABLE table=test-table [id=17df69a4805c4702b81589d6515949d1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 02:52:24.590938 21340 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ea888b104dd0418f974ddcb758c07237. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 02:52:24.592147 21654 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:52:24.584406 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.138.193:41461 (local address 127.0.0.1:34066)
0811 02:52:24.584543 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0811 02:52:24.584561 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 02:52:24.584648 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:52:24.588514 (+ 3866us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:52:24.588517 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:52:24.588536 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 02:52:24.588635 (+ 99us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:52:24.588642 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:52:24.589751 (+ 1109us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:52:24.589754 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:52:24.590778 (+ 1024us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:52:24.590789 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:52:24.591724 (+ 935us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:52:24.591728 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:52:24.591822 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:52:24.591850 (+ 28us) client_negotiation.cc:769] Sending connection context
0811 02:52:24.591933 (+ 83us) client_negotiation.cc:240] Negotiation successful
0811 02:52:24.592016 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":48}
I20260811 02:52:24.592214 21672 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:52:24.584573 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34066 (local address 127.12.138.193:41461)
0811 02:52:24.588251 (+ 3678us) server_negotiation.cc:206] Beginning negotiation
0811 02:52:24.588257 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 02:52:24.588276 (+ 19us) server_negotiation.cc:407] Connection header received
0811 02:52:24.588343 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:52:24.588348 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:52:24.588398 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:52:24.588503 (+ 105us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 02:52:24.588780 (+ 277us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:52:24.589623 (+ 843us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:52:24.590902 (+ 1279us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:52:24.591588 (+ 686us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:52:24.591715 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:52:24.591766 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0811 02:52:24.591974 (+ 208us) server_negotiation.cc:299] Negotiation successful
0811 02:52:24.592069 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3567,"thread_start_us":124,"threads_started":1}
I20260811 02:52:24.592700 21185 tablet_service.cc:1467] Processing CreateTablet for tablet ea888b104dd0418f974ddcb758c07237 (DEFAULT_TABLE table=test-table [id=17df69a4805c4702b81589d6515949d1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 02:52:24.593040 21185 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ea888b104dd0418f974ddcb758c07237. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 02:52:24.593472 21657 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:52:24.583721 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.138.195:42681 (local address 127.0.0.1:36450)
0811 02:52:24.583836 (+ 115us) negotiation.cc:107] Waiting for socket to connect
0811 02:52:24.583852 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 02:52:24.583934 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:52:24.589059 (+ 5125us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:52:24.589062 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:52:24.589080 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 02:52:24.589200 (+ 120us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:52:24.589206 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:52:24.590579 (+ 1373us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:52:24.590583 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:52:24.591868 (+ 1285us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:52:24.591878 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:52:24.593019 (+ 1141us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:52:24.593023 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:52:24.593122 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:52:24.593184 (+ 62us) client_negotiation.cc:769] Sending connection context
0811 02:52:24.593270 (+ 86us) client_negotiation.cc:240] Negotiation successful
0811 02:52:24.593355 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":44}
I20260811 02:52:24.593542 21670 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:52:24.583888 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36450 (local address 127.12.138.195:42681)
0811 02:52:24.588779 (+ 4891us) server_negotiation.cc:206] Beginning negotiation
0811 02:52:24.588794 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0811 02:52:24.588811 (+ 17us) server_negotiation.cc:407] Connection header received
0811 02:52:24.588864 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:52:24.588867 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:52:24.588919 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:52:24.589328 (+ 409us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 02:52:24.589446 (+ 118us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:52:24.590442 (+ 996us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:52:24.592051 (+ 1609us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:52:24.592836 (+ 785us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:52:24.593018 (+ 182us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:52:24.593079 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0811 02:52:24.593289 (+ 210us) server_negotiation.cc:299] Negotiation successful
0811 02:52:24.593389 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4778,"thread_start_us":79,"threads_started":1}
I20260811 02:52:24.593873 21676 tablet_bootstrap.cc:492] T ea888b104dd0418f974ddcb758c07237 P c51f9680930c4419a91d0d0f521006dd: Bootstrap starting.
I20260811 02:52:24.594964 21676 tablet_bootstrap.cc:654] T ea888b104dd0418f974ddcb758c07237 P c51f9680930c4419a91d0d0f521006dd: Neither blocks nor log segments found. Creating new log.
I20260811 02:52:24.596045 21678 tablet_bootstrap.cc:492] T ea888b104dd0418f974ddcb758c07237 P ab096c76c5544f828e9eb7f2bee1977e: Bootstrap starting.
I20260811 02:52:24.596392 21676 tablet_bootstrap.cc:492] T ea888b104dd0418f974ddcb758c07237 P c51f9680930c4419a91d0d0f521006dd: No bootstrap required, opened a new log
I20260811 02:52:24.596505 21676 ts_tablet_manager.cc:1397] T ea888b104dd0418f974ddcb758c07237 P c51f9680930c4419a91d0d0f521006dd: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260811 02:52:24.597002 21678 tablet_bootstrap.cc:654] T ea888b104dd0418f974ddcb758c07237 P ab096c76c5544f828e9eb7f2bee1977e: Neither blocks nor log segments found. Creating new log.
I20260811 02:52:24.596946 21499 tablet_service.cc:1467] Processing CreateTablet for tablet ea888b104dd0418f974ddcb758c07237 (DEFAULT_TABLE table=test-table [id=17df69a4805c4702b81589d6515949d1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 02:52:24.597008 21676 raft_consensus.cc:348] T ea888b104dd0418f974ddcb758c07237 P c51f9680930c4419a91d0d0f521006dd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "adde0ab9117a451c8cf06b46870c7c07" member_type: VOTER last_known_addr { host: "127.12.138.195" port: 42681 } } peers { permanent_uuid: "ab096c76c5544f828e9eb7f2bee1977e" member_type: VOTER last_known_addr { host: "127.12.138.193" port: 41461 } } peers { permanent_uuid: "c51f9680930c4419a91d0d0f521006dd" member_type: VOTER last_known_addr { host: "127.12.138.194" port: 40543 } }
I20260811 02:52:24.597133 21676 raft_consensus.cc:374] T ea888b104dd0418f974ddcb758c07237 P c51f9680930c4419a91d0d0f521006dd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 02:52:24.597240 21676 raft_consensus.cc:729] T ea888b104dd0418f974ddcb758c07237 P c51f9680930c4419a91d0d0f521006dd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c51f9680930c4419a91d0d0f521006dd, State: Initialized, Role: FOLLOWER
I20260811 02:52:24.597282 21499 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ea888b104dd0418f974ddcb758c07237. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 02:52:24.597424 21676 consensus_queue.cc:260] T ea888b104dd0418f974ddcb758c07237 P c51f9680930c4419a91d0d0f521006dd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "adde0ab9117a451c8cf06b46870c7c07" member_type: VOTER last_known_addr { host: "127.12.138.195" port: 42681 } } peers { permanent_uuid: "ab096c76c5544f828e9eb7f2bee1977e" member_type: VOTER last_known_addr { host: "127.12.138.193" port: 41461 } } peers { permanent_uuid: "c51f9680930c4419a91d0d0f521006dd" member_type: VOTER last_known_addr { host: "127.12.138.194" port: 40543 } }
I20260811 02:52:24.597816 21676 ts_tablet_manager.cc:1428] T ea888b104dd0418f974ddcb758c07237 P c51f9680930c4419a91d0d0f521006dd: Time spent starting tablet: real 0.001s user 0.000s sys 0.002s
I20260811 02:52:24.598865 21681 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:52:24.598112 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.138.254:43457 (local address 127.12.138.194:58257)
0811 02:52:24.598676 (+ 564us) negotiation.cc:107] Waiting for socket to connect
0811 02:52:24.598742 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.138.254:43457: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":471,"thread_start_us":98,"threads_started":1}
I20260811 02:52:24.599607 21681 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:52:24.599334 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.138.254:43457 (local address 127.12.138.194:48347)
0811 02:52:24.599475 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0811 02:52:24.599539 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.138.254:43457: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":64}
I20260811 02:52:24.599974 21678 tablet_bootstrap.cc:492] T ea888b104dd0418f974ddcb758c07237 P ab096c76c5544f828e9eb7f2bee1977e: No bootstrap required, opened a new log
I20260811 02:52:24.600060 21678 ts_tablet_manager.cc:1397] T ea888b104dd0418f974ddcb758c07237 P ab096c76c5544f828e9eb7f2bee1977e: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20260811 02:52:24.600404 21420 heartbeater.cc:503] Master 127.12.138.253:42467 was elected leader, sending a full tablet report...
I20260811 02:52:24.600486 21678 raft_consensus.cc:348] T ea888b104dd0418f974ddcb758c07237 P ab096c76c5544f828e9eb7f2bee1977e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "adde0ab9117a451c8cf06b46870c7c07" member_type: VOTER last_known_addr { host: "127.12.138.195" port: 42681 } } peers { permanent_uuid: "ab096c76c5544f828e9eb7f2bee1977e" member_type: VOTER last_known_addr { host: "127.12.138.193" port: 41461 } } peers { permanent_uuid: "c51f9680930c4419a91d0d0f521006dd" member_type: VOTER last_known_addr { host: "127.12.138.194" port: 40543 } }
I20260811 02:52:24.600596 21678 raft_consensus.cc:374] T ea888b104dd0418f974ddcb758c07237 P ab096c76c5544f828e9eb7f2bee1977e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 02:52:24.600627 21678 raft_consensus.cc:729] T ea888b104dd0418f974ddcb758c07237 P ab096c76c5544f828e9eb7f2bee1977e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ab096c76c5544f828e9eb7f2bee1977e, State: Initialized, Role: FOLLOWER
I20260811 02:52:24.600931 21678 consensus_queue.cc:260] T ea888b104dd0418f974ddcb758c07237 P ab096c76c5544f828e9eb7f2bee1977e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "adde0ab9117a451c8cf06b46870c7c07" member_type: VOTER last_known_addr { host: "127.12.138.195" port: 42681 } } peers { permanent_uuid: "ab096c76c5544f828e9eb7f2bee1977e" member_type: VOTER last_known_addr { host: "127.12.138.193" port: 41461 } } peers { permanent_uuid: "c51f9680930c4419a91d0d0f521006dd" member_type: VOTER last_known_addr { host: "127.12.138.194" port: 40543 } }
I20260811 02:52:24.601337 21678 ts_tablet_manager.cc:1428] T ea888b104dd0418f974ddcb758c07237 P ab096c76c5544f828e9eb7f2bee1977e: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260811 02:52:24.602005 21682 tablet_bootstrap.cc:492] T ea888b104dd0418f974ddcb758c07237 P adde0ab9117a451c8cf06b46870c7c07: Bootstrap starting.
I20260811 02:52:24.602811 21684 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:52:24.601734 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.138.254:43457 (local address 127.12.138.193:39649)
0811 02:52:24.602628 (+ 894us) negotiation.cc:107] Waiting for socket to connect
0811 02:52:24.602704 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.138.254:43457: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":830,"thread_start_us":115,"threads_started":1}
I20260811 02:52:24.603152 21682 tablet_bootstrap.cc:654] T ea888b104dd0418f974ddcb758c07237 P adde0ab9117a451c8cf06b46870c7c07: Neither blocks nor log segments found. Creating new log.
I20260811 02:52:24.603319 21262 heartbeater.cc:503] Master 127.12.138.253:42467 was elected leader, sending a full tablet report...
I20260811 02:52:24.603657 21685 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:52:24.603231 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.138.254:43457 (local address 127.12.138.193:33167)
0811 02:52:24.603524 (+ 293us) negotiation.cc:107] Waiting for socket to connect
0811 02:52:24.603576 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.138.254:43457: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":232,"thread_start_us":128,"threads_started":1}
I20260811 02:52:24.605346 21682 tablet_bootstrap.cc:492] T ea888b104dd0418f974ddcb758c07237 P adde0ab9117a451c8cf06b46870c7c07: No bootstrap required, opened a new log
I20260811 02:52:24.605448 21682 ts_tablet_manager.cc:1397] T ea888b104dd0418f974ddcb758c07237 P adde0ab9117a451c8cf06b46870c7c07: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20260811 02:52:24.605839 21682 raft_consensus.cc:348] T ea888b104dd0418f974ddcb758c07237 P adde0ab9117a451c8cf06b46870c7c07 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "adde0ab9117a451c8cf06b46870c7c07" member_type: VOTER last_known_addr { host: "127.12.138.195" port: 42681 } } peers { permanent_uuid: "ab096c76c5544f828e9eb7f2bee1977e" member_type: VOTER last_known_addr { host: "127.12.138.193" port: 41461 } } peers { permanent_uuid: "c51f9680930c4419a91d0d0f521006dd" member_type: VOTER last_known_addr { host: "127.12.138.194" port: 40543 } }
I20260811 02:52:24.605939 21682 raft_consensus.cc:374] T ea888b104dd0418f974ddcb758c07237 P adde0ab9117a451c8cf06b46870c7c07 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 02:52:24.605964 21682 raft_consensus.cc:729] T ea888b104dd0418f974ddcb758c07237 P adde0ab9117a451c8cf06b46870c7c07 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: adde0ab9117a451c8cf06b46870c7c07, State: Initialized, Role: FOLLOWER
I20260811 02:52:24.606070 21682 consensus_queue.cc:260] T ea888b104dd0418f974ddcb758c07237 P adde0ab9117a451c8cf06b46870c7c07 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "adde0ab9117a451c8cf06b46870c7c07" member_type: VOTER last_known_addr { host: "127.12.138.195" port: 42681 } } peers { permanent_uuid: "ab096c76c5544f828e9eb7f2bee1977e" member_type: VOTER last_known_addr { host: "127.12.138.193" port: 41461 } } peers { permanent_uuid: "c51f9680930c4419a91d0d0f521006dd" member_type: VOTER last_known_addr { host: "127.12.138.194" port: 40543 } }
I20260811 02:52:24.606413 21682 ts_tablet_manager.cc:1428] T ea888b104dd0418f974ddcb758c07237 P adde0ab9117a451c8cf06b46870c7c07: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260811 02:52:24.607192 21689 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:52:24.606666 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.138.254:43457 (local address 127.12.138.195:56261)
0811 02:52:24.607014 (+ 348us) negotiation.cc:107] Waiting for socket to connect
0811 02:52:24.607077 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.138.254:43457: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":282,"thread_start_us":102,"threads_started":1}
I20260811 02:52:24.607905 21575 heartbeater.cc:503] Master 127.12.138.253:42467 was elected leader, sending a full tablet report...
I20260811 02:52:24.607991 21689 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:52:24.607662 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.138.254:43457 (local address 127.12.138.195:58063)
0811 02:52:24.607814 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0811 02:52:24.607868 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.138.254:43457: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":75}
W20260811 02:52:24.630056 21263 tablet.cc:2369] T ea888b104dd0418f974ddcb758c07237 P ab096c76c5544f828e9eb7f2bee1977e: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 02:52:24.709120 21680 raft_consensus.cc:482] T ea888b104dd0418f974ddcb758c07237 P c51f9680930c4419a91d0d0f521006dd [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 02:52:24.709400 21680 raft_consensus.cc:504] T ea888b104dd0418f974ddcb758c07237 P c51f9680930c4419a91d0d0f521006dd [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "adde0ab9117a451c8cf06b46870c7c07" member_type: VOTER last_known_addr { host: "127.12.138.195" port: 42681 } } peers { permanent_uuid: "ab096c76c5544f828e9eb7f2bee1977e" member_type: VOTER last_known_addr { host: "127.12.138.193" port: 41461 } } peers { permanent_uuid: "c51f9680930c4419a91d0d0f521006dd" member_type: VOTER last_known_addr { host: "127.12.138.194" port: 40543 } }
I20260811 02:52:24.710819 21680 leader_election.cc:288] T ea888b104dd0418f974ddcb758c07237 P c51f9680930c4419a91d0d0f521006dd [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers adde0ab9117a451c8cf06b46870c7c07 (127.12.138.195:42681), ab096c76c5544f828e9eb7f2bee1977e (127.12.138.193:41461)
I20260811 02:52:24.714803 21672 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:52:24.711177 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.138.194:43161 (local address 127.12.138.193:41461)
0811 02:52:24.711296 (+ 119us) server_negotiation.cc:206] Beginning negotiation
0811 02:52:24.711301 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 02:52:24.711568 (+ 267us) server_negotiation.cc:407] Connection header received
0811 02:52:24.711699 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:52:24.711702 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:52:24.711746 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:52:24.711864 (+ 118us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 02:52:24.712137 (+ 273us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:52:24.712650 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:52:24.713765 (+ 1115us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:52:24.714159 (+ 394us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:52:24.714249 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:52:24.714330 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0811 02:52:24.714570 (+ 240us) server_negotiation.cc:299] Negotiation successful
0811 02:52:24.714653 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260811 02:52:24.714805 21690 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:52:24.711052 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.138.193:41461 (local address 127.12.138.194:43161)
0811 02:52:24.711467 (+ 415us) negotiation.cc:107] Waiting for socket to connect
0811 02:52:24.711481 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 02:52:24.711585 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:52:24.711874 (+ 289us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:52:24.711876 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:52:24.711891 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 02:52:24.712019 (+ 128us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:52:24.712026 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:52:24.712762 (+ 736us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:52:24.712765 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:52:24.713636 (+ 871us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:52:24.713645 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:52:24.714306 (+ 661us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:52:24.714311 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:52:24.714412 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:52:24.714456 (+ 44us) client_negotiation.cc:769] Sending connection context
0811 02:52:24.714518 (+ 62us) client_negotiation.cc:240] Negotiation successful
0811 02:52:24.714622 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":346,"mutex_wait_us":26,"thread_start_us":134,"threads_started":1}
I20260811 02:52:24.714847 21670 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:52:24.711131 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.138.194:46541 (local address 127.12.138.195:42681)
0811 02:52:24.711263 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0811 02:52:24.711268 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 02:52:24.711285 (+ 17us) server_negotiation.cc:407] Connection header received
0811 02:52:24.711482 (+ 197us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:52:24.711487 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:52:24.711551 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:52:24.711635 (+ 84us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 02:52:24.711975 (+ 340us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:52:24.712549 (+ 574us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:52:24.713798 (+ 1249us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:52:24.714189 (+ 391us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:52:24.714331 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:52:24.714385 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0811 02:52:24.714644 (+ 259us) server_negotiation.cc:299] Negotiation successful
0811 02:52:24.714719 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260811 02:52:24.714811 21681 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:52:24.711022 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.138.195:42681 (local address 127.12.138.194:46541)
0811 02:52:24.711204 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0811 02:52:24.711220 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 02:52:24.711338 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:52:24.711676 (+ 338us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:52:24.711679 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:52:24.711698 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 02:52:24.711848 (+ 150us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:52:24.711858 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:52:24.712665 (+ 807us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:52:24.712669 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:52:24.713639 (+ 970us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:52:24.713648 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:52:24.714347 (+ 699us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:52:24.714353 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:52:24.714471 (+ 118us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:52:24.714489 (+ 18us) client_negotiation.cc:769] Sending connection context
0811 02:52:24.714607 (+ 118us) client_negotiation.cc:240] Negotiation successful
0811 02:52:24.714699 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":104,"mutex_wait_us":41}
I20260811 02:52:24.715324 21518 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ea888b104dd0418f974ddcb758c07237" candidate_uuid: "c51f9680930c4419a91d0d0f521006dd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "adde0ab9117a451c8cf06b46870c7c07" is_pre_election: true
I20260811 02:52:24.715389 21207 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ea888b104dd0418f974ddcb758c07237" candidate_uuid: "c51f9680930c4419a91d0d0f521006dd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ab096c76c5544f828e9eb7f2bee1977e" is_pre_election: true
I20260811 02:52:24.715502 21518 raft_consensus.cc:2454] T ea888b104dd0418f974ddcb758c07237 P adde0ab9117a451c8cf06b46870c7c07 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c51f9680930c4419a91d0d0f521006dd in term 0.
I20260811 02:52:24.715647 21207 raft_consensus.cc:2454] T ea888b104dd0418f974ddcb758c07237 P ab096c76c5544f828e9eb7f2bee1977e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c51f9680930c4419a91d0d0f521006dd in term 0.
I20260811 02:52:24.715914 21288 leader_election.cc:302] T ea888b104dd0418f974ddcb758c07237 P c51f9680930c4419a91d0d0f521006dd [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: adde0ab9117a451c8cf06b46870c7c07, c51f9680930c4419a91d0d0f521006dd; no voters:
I20260811 02:52:24.716177 21680 raft_consensus.cc:2781] T ea888b104dd0418f974ddcb758c07237 P c51f9680930c4419a91d0d0f521006dd [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 02:52:24.716235 21680 raft_consensus.cc:482] T ea888b104dd0418f974ddcb758c07237 P c51f9680930c4419a91d0d0f521006dd [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 02:52:24.716262 21680 raft_consensus.cc:3037] T ea888b104dd0418f974ddcb758c07237 P c51f9680930c4419a91d0d0f521006dd [term 0 FOLLOWER]: Advancing to term 1
I20260811 02:52:24.717384 21680 raft_consensus.cc:504] T ea888b104dd0418f974ddcb758c07237 P c51f9680930c4419a91d0d0f521006dd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "adde0ab9117a451c8cf06b46870c7c07" member_type: VOTER last_known_addr { host: "127.12.138.195" port: 42681 } } peers { permanent_uuid: "ab096c76c5544f828e9eb7f2bee1977e" member_type: VOTER last_known_addr { host: "127.12.138.193" port: 41461 } } peers { permanent_uuid: "c51f9680930c4419a91d0d0f521006dd" member_type: VOTER last_known_addr { host: "127.12.138.194" port: 40543 } }
I20260811 02:52:24.717751 21680 leader_election.cc:288] T ea888b104dd0418f974ddcb758c07237 P c51f9680930c4419a91d0d0f521006dd [CANDIDATE]: Term 1 election: Requested vote from peers adde0ab9117a451c8cf06b46870c7c07 (127.12.138.195:42681), ab096c76c5544f828e9eb7f2bee1977e (127.12.138.193:41461)
I20260811 02:52:24.718096 21518 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ea888b104dd0418f974ddcb758c07237" candidate_uuid: "c51f9680930c4419a91d0d0f521006dd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "adde0ab9117a451c8cf06b46870c7c07"
I20260811 02:52:24.718091 21207 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ea888b104dd0418f974ddcb758c07237" candidate_uuid: "c51f9680930c4419a91d0d0f521006dd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ab096c76c5544f828e9eb7f2bee1977e"
I20260811 02:52:24.718227 21207 raft_consensus.cc:3037] T ea888b104dd0418f974ddcb758c07237 P ab096c76c5544f828e9eb7f2bee1977e [term 0 FOLLOWER]: Advancing to term 1
I20260811 02:52:24.718227 21518 raft_consensus.cc:3037] T ea888b104dd0418f974ddcb758c07237 P adde0ab9117a451c8cf06b46870c7c07 [term 0 FOLLOWER]: Advancing to term 1
I20260811 02:52:24.719385 21518 raft_consensus.cc:2454] T ea888b104dd0418f974ddcb758c07237 P adde0ab9117a451c8cf06b46870c7c07 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c51f9680930c4419a91d0d0f521006dd in term 1.
I20260811 02:52:24.719447 21207 raft_consensus.cc:2454] T ea888b104dd0418f974ddcb758c07237 P ab096c76c5544f828e9eb7f2bee1977e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c51f9680930c4419a91d0d0f521006dd in term 1.
I20260811 02:52:24.719744 21288 leader_election.cc:302] T ea888b104dd0418f974ddcb758c07237 P c51f9680930c4419a91d0d0f521006dd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: adde0ab9117a451c8cf06b46870c7c07, c51f9680930c4419a91d0d0f521006dd; no voters:
I20260811 02:52:24.719913 21680 raft_consensus.cc:2781] T ea888b104dd0418f974ddcb758c07237 P c51f9680930c4419a91d0d0f521006dd [term 1 FOLLOWER]: Leader election won for term 1
I20260811 02:52:24.720198 21680 raft_consensus.cc:686] T ea888b104dd0418f974ddcb758c07237 P c51f9680930c4419a91d0d0f521006dd [term 1 LEADER]: Becoming Leader. State: Replica: c51f9680930c4419a91d0d0f521006dd, State: Running, Role: LEADER
I20260811 02:52:24.720534 21680 consensus_queue.cc:237] T ea888b104dd0418f974ddcb758c07237 P c51f9680930c4419a91d0d0f521006dd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "adde0ab9117a451c8cf06b46870c7c07" member_type: VOTER last_known_addr { host: "127.12.138.195" port: 42681 } } peers { permanent_uuid: "ab096c76c5544f828e9eb7f2bee1977e" member_type: VOTER last_known_addr { host: "127.12.138.193" port: 41461 } } peers { permanent_uuid: "c51f9680930c4419a91d0d0f521006dd" member_type: VOTER last_known_addr { host: "127.12.138.194" port: 40543 } }
I20260811 02:52:24.720989 21690 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:52:24.720672 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.138.254:43457 (local address 127.12.138.194:57409)
0811 02:52:24.720837 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0811 02:52:24.720915 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.138.254:43457: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":82}
I20260811 02:52:24.723235 20830 catalog_manager.cc:5495] T ea888b104dd0418f974ddcb758c07237 P c51f9680930c4419a91d0d0f521006dd reported cstate change: term changed from 0 to 1, leader changed from <none> to c51f9680930c4419a91d0d0f521006dd (127.12.138.194). New cstate: current_term: 1 leader_uuid: "c51f9680930c4419a91d0d0f521006dd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "adde0ab9117a451c8cf06b46870c7c07" member_type: VOTER last_known_addr { host: "127.12.138.195" port: 42681 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ab096c76c5544f828e9eb7f2bee1977e" member_type: VOTER last_known_addr { host: "127.12.138.193" port: 41461 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c51f9680930c4419a91d0d0f521006dd" member_type: VOTER last_known_addr { host: "127.12.138.194" port: 40543 } health_report { overall_health: HEALTHY } } }
I20260811 02:52:24.748487 21674 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:52:24.744868 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52780 (local address 127.12.138.194:40543)
0811 02:52:24.745018 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0811 02:52:24.745025 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 02:52:24.745042 (+ 17us) server_negotiation.cc:407] Connection header received
0811 02:52:24.745131 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:52:24.745133 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:52:24.745203 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:52:24.745332 (+ 129us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 02:52:24.745594 (+ 262us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:52:24.746120 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:52:24.747004 (+ 884us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:52:24.747223 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:52:24.747342 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:52:24.747626 (+ 284us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 02:52:24.748080 (+ 454us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 02:52:24.748215 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0811 02:52:24.748278 (+ 63us) server_negotiation.cc:299] Negotiation successful
0811 02:52:24.748337 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260811 02:52:24.751318 21518 raft_consensus.cc:1261] T ea888b104dd0418f974ddcb758c07237 P adde0ab9117a451c8cf06b46870c7c07 [term 1 FOLLOWER]: Refusing update from remote peer c51f9680930c4419a91d0d0f521006dd: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 02:52:24.751590 21207 raft_consensus.cc:1261] T ea888b104dd0418f974ddcb758c07237 P ab096c76c5544f828e9eb7f2bee1977e [term 1 FOLLOWER]: Refusing update from remote peer c51f9680930c4419a91d0d0f521006dd: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 02:52:24.751888 21680 consensus_queue.cc:1035] T ea888b104dd0418f974ddcb758c07237 P c51f9680930c4419a91d0d0f521006dd [LEADER]: Connected to new peer: Peer: permanent_uuid: "adde0ab9117a451c8cf06b46870c7c07" member_type: VOTER last_known_addr { host: "127.12.138.195" port: 42681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 02:52:24.752183 21691 consensus_queue.cc:1035] T ea888b104dd0418f974ddcb758c07237 P c51f9680930c4419a91d0d0f521006dd [LEADER]: Connected to new peer: Peer: permanent_uuid: "ab096c76c5544f828e9eb7f2bee1977e" member_type: VOTER last_known_addr { host: "127.12.138.193" port: 41461 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 02:52:24.755949 21684 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:52:24.755628 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.138.254:43457 (local address 127.12.138.193:35009)
0811 02:52:24.755755 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0811 02:52:24.755852 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.138.254:43457: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
I20260811 02:52:24.758208 21689 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:52:24.754254 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.138.254:43457 (local address 127.12.138.195:58375)
0811 02:52:24.758043 (+ 3789us) negotiation.cc:107] Waiting for socket to connect
0811 02:52:24.758115 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.138.254:43457: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":3701}
I20260811 02:52:24.758651 21690 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:52:24.758200 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.138.254:43457 (local address 127.12.138.194:44269)
0811 02:52:24.758513 (+ 313us) negotiation.cc:107] Waiting for socket to connect
0811 02:52:24.758580 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.138.254:43457: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":238}
I20260811 02:52:24.759995 21684 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:52:24.759614 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.138.254:43457 (local address 127.12.138.193:49019)
0811 02:52:24.759851 (+ 237us) negotiation.cc:107] Waiting for socket to connect
0811 02:52:24.759917 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.138.254:43457: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":149}
I20260811 02:52:24.760548 21690 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:52:24.760254 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.138.254:43457 (local address 127.12.138.194:56713)
0811 02:52:24.760415 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0811 02:52:24.760464 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.138.254:43457: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":92}
I20260811 02:52:24.762228 21689 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:52:24.759227 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.138.254:43457 (local address 127.12.138.195:49279)
0811 02:52:24.762104 (+ 2877us) negotiation.cc:107] Waiting for socket to connect
0811 02:52:24.762159 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.138.254:43457: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":2128}
I20260811 02:52:24.768078 20837 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44246:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 02:52:24.768256 20837 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44246:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 02:52:24.772401 20837 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 269d7f529b994b32bc6d6d1b512371d4: Sending DeleteTablet for 3 replicas of tablet ea888b104dd0418f974ddcb758c07237
I20260811 02:52:24.773118 21499 tablet_service.cc:1514] Processing DeleteTablet for tablet ea888b104dd0418f974ddcb758c07237 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 02:52:24 UTC) from {username='oryx', principal='oryx/127.12.138.253@KRBTEST.COM'} at 127.0.0.1:36450
I20260811 02:52:24.773558 21705 tablet_replica.cc:330] T ea888b104dd0418f974ddcb758c07237 P adde0ab9117a451c8cf06b46870c7c07: stopping tablet replica
I20260811 02:52:24.773799 21705 raft_consensus.cc:2229] T ea888b104dd0418f974ddcb758c07237 P adde0ab9117a451c8cf06b46870c7c07 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 02:52:24.773957 21340 tablet_service.cc:1514] Processing DeleteTablet for tablet ea888b104dd0418f974ddcb758c07237 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 02:52:24 UTC) from {username='oryx', principal='oryx/127.12.138.253@KRBTEST.COM'} at 127.0.0.1:52778
I20260811 02:52:24.774026 21705 raft_consensus.cc:2258] T ea888b104dd0418f974ddcb758c07237 P adde0ab9117a451c8cf06b46870c7c07 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 02:52:24.774322 21185 tablet_service.cc:1514] Processing DeleteTablet for tablet ea888b104dd0418f974ddcb758c07237 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 02:52:24 UTC) from {username='oryx', principal='oryx/127.12.138.253@KRBTEST.COM'} at 127.0.0.1:34066
I20260811 02:52:24.774398 21707 tablet_replica.cc:330] T ea888b104dd0418f974ddcb758c07237 P c51f9680930c4419a91d0d0f521006dd: stopping tablet replica
I20260811 02:52:24.774507 12843 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDawEwS/build/debug/bin/kudu with pid 21004
I20260811 02:52:24.777426 21705 ts_tablet_manager.cc:1905] T ea888b104dd0418f974ddcb758c07237 P adde0ab9117a451c8cf06b46870c7c07: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 02:52:24.774518 21707 raft_consensus.cc:2229] T ea888b104dd0418f974ddcb758c07237 P c51f9680930c4419a91d0d0f521006dd [term 1 LEADER]: Raft consensus shutting down.
I20260811 02:52:24.781616 21707 raft_consensus.cc:2258] T ea888b104dd0418f974ddcb758c07237 P c51f9680930c4419a91d0d0f521006dd [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 02:52:24.781855 21705 ts_tablet_manager.cc:1918] T ea888b104dd0418f974ddcb758c07237 P adde0ab9117a451c8cf06b46870c7c07: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 02:52:24.782004 21705 log.cc:1196] T ea888b104dd0418f974ddcb758c07237 P adde0ab9117a451c8cf06b46870c7c07: Deleting WAL directory at /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786416707705494-12843-0/minicluster-data/ts-2/wal/wals/ea888b104dd0418f974ddcb758c07237
I20260811 02:52:24.782418 21705 ts_tablet_manager.cc:1939] T ea888b104dd0418f974ddcb758c07237 P adde0ab9117a451c8cf06b46870c7c07: Deleting consensus metadata
I20260811 02:52:24.782939 21707 ts_tablet_manager.cc:1905] T ea888b104dd0418f974ddcb758c07237 P c51f9680930c4419a91d0d0f521006dd: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 02:52:24.783779 20822 catalog_manager.cc:4841] TS adde0ab9117a451c8cf06b46870c7c07 (127.12.138.195:42681): tablet ea888b104dd0418f974ddcb758c07237 (table test-table [id=17df69a4805c4702b81589d6515949d1]) successfully deleted
I20260811 02:52:24.785799 21707 ts_tablet_manager.cc:1918] T ea888b104dd0418f974ddcb758c07237 P c51f9680930c4419a91d0d0f521006dd: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 02:52:24.785913 21707 log.cc:1196] T ea888b104dd0418f974ddcb758c07237 P c51f9680930c4419a91d0d0f521006dd: Deleting WAL directory at /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786416707705494-12843-0/minicluster-data/ts-1/wal/wals/ea888b104dd0418f974ddcb758c07237
I20260811 02:52:24.786181 21707 ts_tablet_manager.cc:1939] T ea888b104dd0418f974ddcb758c07237 P c51f9680930c4419a91d0d0f521006dd: Deleting consensus metadata
I20260811 02:52:24.786868 20824 catalog_manager.cc:4841] TS c51f9680930c4419a91d0d0f521006dd (127.12.138.194:40543): tablet ea888b104dd0418f974ddcb758c07237 (table test-table [id=17df69a4805c4702b81589d6515949d1]) successfully deleted
I20260811 02:52:24.793031 12843 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDawEwS/build/debug/bin/kudu with pid 21269
I20260811 02:52:24.796466 21657 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:52:24.795930 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.138.193:41461 (local address 127.0.0.1:34082)
0811 02:52:24.796234 (+ 304us) negotiation.cc:107] Waiting for socket to connect
0811 02:52:24.796356 (+ 122us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.138.193:41461: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":176}
W20260811 02:52:24.796706 20824 catalog_manager.cc:4584] TS ab096c76c5544f828e9eb7f2bee1977e (127.12.138.193:41461): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet ea888b104dd0418f974ddcb758c07237: Network error: Client connection negotiation failed: client connection to 127.12.138.193:41461: connect: Connection refused (error 111)
I20260811 02:52:24.803690 12843 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDawEwS/build/debug/bin/kudu with pid 21426
I20260811 02:52:24.813421 12843 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDawEwS/build/debug/bin/kudu with pid 20683
I20260811 02:52:24.822443 12843 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDawEwS/build/debug/bin/kudu with pid 20805
2026-08-11T02:52:24Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipalMultipleMaster (5578 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786416707705494-12843-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 02:52:24 dist-test-slave-9gcw krb5kdc[21711](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 02:52:24 dist-test-slave-9gcw krb5kdc[21711](info): set up 2 sockets
Aug 11 02:52:24 dist-test-slave-9gcw krb5kdc[21711](info): commencing operation
krb5kdc: starting...
W20260811 02:52:25.967209 12843 mini_kdc.cc:121] Time spent starting KDC: real 1.119s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786416707705494-12843-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786416707705494-12843-0/krb5kdc/test-user.keytab.
Aug 11 02:52:26 dist-test-slave-9gcw krb5kdc[21711](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416746, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T02:52:26Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T02:52:26Z Disabled control of system clock
WARNING: no policy specified for oryx/127.12.138.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.12.138.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.12.138.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786416707705494-12843-0/krb5kdc/oryx_127.12.138.254.keytab.
Entry for principal oryx/127.12.138.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786416707705494-12843-0/krb5kdc/oryx_127.12.138.254.keytab.
I20260811 02:52:26.108048 12843 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDawEwS/build/debug/bin/kudu
/tmp/dist-test-taskDawEwS/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786416707705494-12843-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786416707705494-12843-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786416707705494-12843-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786416707705494-12843-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.138.254:43149
--webserver_interface=127.12.138.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786416707705494-12843-0/krb5kdc/oryx_127.12.138.254.keytab
--principal=oryx/127.12.138.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.138.212:36165
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.12.138.254:43149
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786416707705494-12843-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786416707705494-12843-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786416707705494-12843-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 02:52:26.227028 21725 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 02:52:26.227340 21725 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 02:52:26.227420 21725 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 02:52:26.231537 21725 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 02:52:26.231624 21725 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260811 02:52:26.231664 21725 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 02:52:26.231729 21725 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 02:52:26.231784 21725 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 02:52:26.231823 21725 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 02:52:26.237339 21725 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.138.212:36165
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786416707705494-12843-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786416707705494-12843-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.12.138.254:43149
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786416707705494-12843-0/krb5kdc/oryx_127.12.138.254.keytab
--principal=oryx/127.12.138.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.138.254:43149
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786416707705494-12843-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.12.138.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21725
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786416707705494-12843-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 02:09:49 UTC on ccycloud.quasar-fsnqqc.root.comops.site
build id 4579570
I20260811 02:52:26.238432 21725 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 02:52:26.239573 21725 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 02:52:26.248569 21731 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:52:26.248564 21732 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:52:26.248755 21725 server_base.cc:1034] running on GCE node
W20260811 02:52:26.248564 21735 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:52:26.249562 21725 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 02:52:26.250691 21725 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 02:52:26.251868 21725 hybrid_clock.cc:648] HybridClock initialized: now 1786416746251849 us; error 34 us; skew 500 ppm
Aug 11 02:52:26 dist-test-slave-9gcw krb5kdc[21711](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416746, etypes {rep=17 tkt=17 ses=17}, oryx/127.12.138.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 02:52:26.255000 21725 init.cc:377] Logged in from keytab as oryx/127.12.138.254@KRBTEST.COM (short username oryx)
I20260811 02:52:26.256000 21725 webserver.cc:460] Webserver started at http://127.12.138.254:42719/ using document root <none> and password file <none>
I20260811 02:52:26.256628 21725 fs_manager.cc:359] Metadata directory not provided
I20260811 02:52:26.256693 21725 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 02:52:26.256932 21725 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 02:52:26.259385 21725 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786416707705494-12843-0/minicluster-data/master-0/data/instance:
uuid: "acf0fa3758a14749a4a0ec3e106b3dd5"
format_stamp: "Formatted at 2026-08-11 02:52:26 on dist-test-slave-9gcw"
server_key: "e4728e924c2b23dc9316b99159e9c20e"
server_key_iv: "f05b8ceb4d725e46e0440726b55aea1a"
server_key_version: "encryptionkey@0"
I20260811 02:52:26.260181 21725 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786416707705494-12843-0/minicluster-data/master-0/wal/instance:
uuid: "acf0fa3758a14749a4a0ec3e106b3dd5"
format_stamp: "Formatted at 2026-08-11 02:52:26 on dist-test-slave-9gcw"
server_key: "e4728e924c2b23dc9316b99159e9c20e"
server_key_iv: "f05b8ceb4d725e46e0440726b55aea1a"
server_key_version: "encryptionkey@0"
I20260811 02:52:26.264436 21725 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.004s
I20260811 02:52:26.267138 21742 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 02:52:26.268560 21725 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260811 02:52:26.268718 21725 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786416707705494-12843-0/minicluster-data/master-0/data,/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786416707705494-12843-0/minicluster-data/master-0/wal
uuid: "acf0fa3758a14749a4a0ec3e106b3dd5"
format_stamp: "Formatted at 2026-08-11 02:52:26 on dist-test-slave-9gcw"
server_key: "e4728e924c2b23dc9316b99159e9c20e"
server_key_iv: "f05b8ceb4d725e46e0440726b55aea1a"
server_key_version: "encryptionkey@0"
I20260811 02:52:26.268852 21725 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786416707705494-12843-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786416707705494-12843-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786416707705494-12843-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 02:52:26.284281 21725 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 02:52:26.287779 21725 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 02:52:26.288012 21725 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 02:52:26.296290 21725 rpc_server.cc:307] RPC server started. Bound to: 127.12.138.254:43149
I20260811 02:52:26.296314 21812 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.138.254:43149 every 8 connection(s)
I20260811 02:52:26.297353 21725 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786416707705494-12843-0/minicluster-data/master-0/data/info.pb
I20260811 02:52:26.300215 21813 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 02:52:26.305828 12843 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDawEwS/build/debug/bin/kudu as pid 21725
I20260811 02:52:26.305938 12843 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786416707705494-12843-0/minicluster-data/master-0/wal/instance
I20260811 02:52:26.306089 21813 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P acf0fa3758a14749a4a0ec3e106b3dd5: Bootstrap starting.
I20260811 02:52:26.306259 12843 external_mini_cluster.cc:1428] Setting key ce58a4b8660109f6b93c93bb73c3e824
I20260811 02:52:26.308975 21813 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P acf0fa3758a14749a4a0ec3e106b3dd5: Neither blocks nor log segments found. Creating new log.
I20260811 02:52:26.309942 21813 log.cc:824] T 00000000000000000000000000000000 P acf0fa3758a14749a4a0ec3e106b3dd5: Log is configured to *not* fsync() on all Append() calls
I20260811 02:52:26.312340 21813 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P acf0fa3758a14749a4a0ec3e106b3dd5: No bootstrap required, opened a new log
Aug 11 02:52:26 dist-test-slave-9gcw krb5kdc[21711](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416746, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.12.138.254@KRBTEST.COM
I20260811 02:52:26.316712 21813 raft_consensus.cc:348] T 00000000000000000000000000000000 P acf0fa3758a14749a4a0ec3e106b3dd5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "acf0fa3758a14749a4a0ec3e106b3dd5" member_type: VOTER last_known_addr { host: "127.12.138.254" port: 43149 } }
I20260811 02:52:26.316993 21813 raft_consensus.cc:374] T 00000000000000000000000000000000 P acf0fa3758a14749a4a0ec3e106b3dd5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 02:52:26.317075 21813 raft_consensus.cc:729] T 00000000000000000000000000000000 P acf0fa3758a14749a4a0ec3e106b3dd5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: acf0fa3758a14749a4a0ec3e106b3dd5, State: Initialized, Role: FOLLOWER
I20260811 02:52:26.317636 21813 consensus_queue.cc:260] T 00000000000000000000000000000000 P acf0fa3758a14749a4a0ec3e106b3dd5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "acf0fa3758a14749a4a0ec3e106b3dd5" member_type: VOTER last_known_addr { host: "127.12.138.254" port: 43149 } }
I20260811 02:52:26.317847 21813 raft_consensus.cc:388] T 00000000000000000000000000000000 P acf0fa3758a14749a4a0ec3e106b3dd5 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 02:52:26.317932 21813 raft_consensus.cc:482] T 00000000000000000000000000000000 P acf0fa3758a14749a4a0ec3e106b3dd5 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 02:52:26.318049 21813 raft_consensus.cc:3037] T 00000000000000000000000000000000 P acf0fa3758a14749a4a0ec3e106b3dd5 [term 0 FOLLOWER]: Advancing to term 1
I20260811 02:52:26.319325 21813 raft_consensus.cc:504] T 00000000000000000000000000000000 P acf0fa3758a14749a4a0ec3e106b3dd5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "acf0fa3758a14749a4a0ec3e106b3dd5" member_type: VOTER last_known_addr { host: "127.12.138.254" port: 43149 } }
I20260811 02:52:26.319726 21813 leader_election.cc:302] T 00000000000000000000000000000000 P acf0fa3758a14749a4a0ec3e106b3dd5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: acf0fa3758a14749a4a0ec3e106b3dd5; no voters:
I20260811 02:52:26.320127 21813 leader_election.cc:288] T 00000000000000000000000000000000 P acf0fa3758a14749a4a0ec3e106b3dd5 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 02:52:26.320240 21818 raft_consensus.cc:2781] T 00000000000000000000000000000000 P acf0fa3758a14749a4a0ec3e106b3dd5 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 02:52:26.320494 21818 raft_consensus.cc:686] T 00000000000000000000000000000000 P acf0fa3758a14749a4a0ec3e106b3dd5 [term 1 LEADER]: Becoming Leader. State: Replica: acf0fa3758a14749a4a0ec3e106b3dd5, State: Running, Role: LEADER
I20260811 02:52:26.320824 21818 consensus_queue.cc:237] T 00000000000000000000000000000000 P acf0fa3758a14749a4a0ec3e106b3dd5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "acf0fa3758a14749a4a0ec3e106b3dd5" member_type: VOTER last_known_addr { host: "127.12.138.254" port: 43149 } }
I20260811 02:52:26.321518 21813 sys_catalog.cc:564] T 00000000000000000000000000000000 P acf0fa3758a14749a4a0ec3e106b3dd5 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 02:52:26.322155 21818 sys_catalog.cc:455] T 00000000000000000000000000000000 P acf0fa3758a14749a4a0ec3e106b3dd5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader acf0fa3758a14749a4a0ec3e106b3dd5. Latest consensus state: current_term: 1 leader_uuid: "acf0fa3758a14749a4a0ec3e106b3dd5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "acf0fa3758a14749a4a0ec3e106b3dd5" member_type: VOTER last_known_addr { host: "127.12.138.254" port: 43149 } } }
I20260811 02:52:26.322263 21818 sys_catalog.cc:458] T 00000000000000000000000000000000 P acf0fa3758a14749a4a0ec3e106b3dd5 [sys.catalog]: This master's current role is: LEADER
I20260811 02:52:26.322649 21819 sys_catalog.cc:455] T 00000000000000000000000000000000 P acf0fa3758a14749a4a0ec3e106b3dd5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "acf0fa3758a14749a4a0ec3e106b3dd5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "acf0fa3758a14749a4a0ec3e106b3dd5" member_type: VOTER last_known_addr { host: "127.12.138.254" port: 43149 } } }
I20260811 02:52:26.322747 21819 sys_catalog.cc:458] T 00000000000000000000000000000000 P acf0fa3758a14749a4a0ec3e106b3dd5 [sys.catalog]: This master's current role is: LEADER
I20260811 02:52:26.323076 21816 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:52:26.307710 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36650 (local address 127.12.138.254:43149)
0811 02:52:26.308191 (+ 481us) server_negotiation.cc:206] Beginning negotiation
0811 02:52:26.308212 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0811 02:52:26.308257 (+ 45us) server_negotiation.cc:407] Connection header received
0811 02:52:26.309229 (+ 972us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:52:26.309260 (+ 31us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:52:26.309705 (+ 445us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:52:26.310087 (+ 382us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:52:26.311046 (+ 959us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:52:26.312579 (+ 1533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:52:26.313566 (+ 987us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:52:26.314365 (+ 799us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:52:26.314532 (+ 167us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:52:26.317361 (+ 2829us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:52:26.317367 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:52:26.317383 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:52:26.317416 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:52:26.319574 (+ 2158us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:52:26.320065 (+ 491us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:52:26.320074 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:52:26.320081 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:52:26.320234 (+ 153us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:52:26.320705 (+ 471us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:52:26.320708 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:52:26.320710 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:52:26.321257 (+ 547us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:52:26.321472 (+ 215us) server_negotiation.cc:1035] Waiting for connection context
0811 02:52:26.321834 (+ 362us) server_negotiation.cc:299] Negotiation successful
0811 02:52:26.322166 (+ 332us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":324,"thread_start_us":121,"threads_started":1}
I20260811 02:52:26.324553 21824 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 02:52:26.327252 21824 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 02:52:26.332756 21824 catalog_manager.cc:1347] Generated new cluster ID: fcb099c4217641d89fcd16286a5260e4
I20260811 02:52:26.332835 21824 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 02:52:26.344873 21824 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 02:52:26.345991 21824 catalog_manager.cc:1504] Loading token signing keys...
I20260811 02:52:26.352620 21824 catalog_manager.cc:5868] T 00000000000000000000000000000000 P acf0fa3758a14749a4a0ec3e106b3dd5: Generated new TSK 0
I20260811 02:52:26.353564 21824 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.12.138.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.12.138.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.12.138.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786416707705494-12843-0/krb5kdc/oryx_127.12.138.193.keytab.
Entry for principal oryx/127.12.138.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786416707705494-12843-0/krb5kdc/oryx_127.12.138.193.keytab.
I20260811 02:52:26.392138 12843 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDawEwS/build/debug/bin/kudu
/tmp/dist-test-taskDawEwS/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786416707705494-12843-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786416707705494-12843-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786416707705494-12843-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786416707705494-12843-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.138.193:0
--local_ip_for_outbound_sockets=127.12.138.193
--webserver_interface=127.12.138.193
--webserver_port=0
--tserver_master_addrs=127.12.138.254:43149
--keytab_file=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786416707705494-12843-0/krb5kdc/oryx_127.12.138.193.keytab
--principal=oryx/127.12.138.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.138.212:36165
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786416707705494-12843-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786416707705494-12843-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786416707705494-12843-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 02:52:26.510838 21841 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 02:52:26.511189 21841 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 02:52:26.511287 21841 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 02:52:26.515451 21841 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 02:52:26.515560 21841 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 02:52:26.515636 21841 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260811 02:52:26.515719 21841 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.138.193
I20260811 02:52:26.521124 21841 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.138.212:36165
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786416707705494-12843-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786416707705494-12843-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786416707705494-12843-0/krb5kdc/oryx_127.12.138.193.keytab
--principal=oryx/127.12.138.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.138.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786416707705494-12843-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.12.138.193
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.12.138.254:43149
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21841
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.138.193
--log_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786416707705494-12843-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 02:09:49 UTC on ccycloud.quasar-fsnqqc.root.comops.site
build id 4579570
I20260811 02:52:26.522436 21841 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 02:52:26.523579 21841 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 02:52:26.533804 21850 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:52:26.533846 21847 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:52:26.533776 21846 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:52:26.534055 21841 server_base.cc:1034] running on GCE node
I20260811 02:52:26.534528 21841 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 02:52:26.535216 21841 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 02:52:26.536412 21841 hybrid_clock.cc:648] HybridClock initialized: now 1786416746536369 us; error 56 us; skew 500 ppm
Aug 11 02:52:26 dist-test-slave-9gcw krb5kdc[21711](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416746, etypes {rep=17 tkt=17 ses=17}, oryx/127.12.138.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 02:52:26.539520 21841 init.cc:377] Logged in from keytab as oryx/127.12.138.193@KRBTEST.COM (short username oryx)
I20260811 02:52:26.540570 21841 webserver.cc:460] Webserver started at http://127.12.138.193:40931/ using document root <none> and password file <none>
I20260811 02:52:26.541128 21841 fs_manager.cc:359] Metadata directory not provided
I20260811 02:52:26.541254 21841 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 02:52:26.541442 21841 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 02:52:26.543500 21841 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786416707705494-12843-0/minicluster-data/ts-0/data/instance:
uuid: "6937bfa1e57344d78f8cc3b5df278e87"
format_stamp: "Formatted at 2026-08-11 02:52:26 on dist-test-slave-9gcw"
server_key: "b85c39db934a2b99258cfd0a7a095448"
server_key_iv: "419b243b16ead9b6626893a9396d39a5"
server_key_version: "encryptionkey@0"
I20260811 02:52:26.544039 21841 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786416707705494-12843-0/minicluster-data/ts-0/wal/instance:
uuid: "6937bfa1e57344d78f8cc3b5df278e87"
format_stamp: "Formatted at 2026-08-11 02:52:26 on dist-test-slave-9gcw"
server_key: "b85c39db934a2b99258cfd0a7a095448"
server_key_iv: "419b243b16ead9b6626893a9396d39a5"
server_key_version: "encryptionkey@0"
I20260811 02:52:26.547828 21841 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.001s sys 0.005s
I20260811 02:52:26.550437 21858 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 02:52:26.551761 21841 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260811 02:52:26.551903 21841 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786416707705494-12843-0/minicluster-data/ts-0/data,/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786416707705494-12843-0/minicluster-data/ts-0/wal
uuid: "6937bfa1e57344d78f8cc3b5df278e87"
format_stamp: "Formatted at 2026-08-11 02:52:26 on dist-test-slave-9gcw"
server_key: "b85c39db934a2b99258cfd0a7a095448"
server_key_iv: "419b243b16ead9b6626893a9396d39a5"
server_key_version: "encryptionkey@0"
I20260811 02:52:26.552019 21841 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786416707705494-12843-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786416707705494-12843-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786416707705494-12843-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 02:52:26.567623 21841 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 02:52:26.571256 21841 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 02:52:26.571460 21841 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 02:52:26.573210 21841 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 02:52:26.573267 21841 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 02:52:26.573331 21841 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 02:52:26.573364 21841 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 11 02:52:26 dist-test-slave-9gcw krb5kdc[21711](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416746, etypes {rep=17 tkt=17 ses=17}, oryx/127.12.138.193@KRBTEST.COM for oryx/127.12.138.254@KRBTEST.COM
I20260811 02:52:26.584281 21841 rpc_server.cc:307] RPC server started. Bound to: 127.12.138.193:34941
I20260811 02:52:26.584412 21988 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.138.193:34941 every 8 connection(s)
I20260811 02:52:26.585461 21841 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786416707705494-12843-0/minicluster-data/ts-0/data/info.pb
I20260811 02:52:26.588407 21816 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:52:26.574054 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.138.193:50007 (local address 127.12.138.254:43149)
0811 02:52:26.574228 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0811 02:52:26.574235 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 02:52:26.575112 (+ 877us) server_negotiation.cc:407] Connection header received
0811 02:52:26.576199 (+ 1087us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:52:26.576203 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:52:26.576271 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:52:26.576349 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:52:26.578356 (+ 2007us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:52:26.579317 (+ 961us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:52:26.581240 (+ 1923us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:52:26.581619 (+ 379us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:52:26.581763 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:52:26.584841 (+ 3078us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:52:26.584845 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:52:26.584848 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:52:26.584879 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:52:26.586494 (+ 1615us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:52:26.587049 (+ 555us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:52:26.587052 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:52:26.587053 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:52:26.587130 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:52:26.587451 (+ 321us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:52:26.587453 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:52:26.587454 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:52:26.587631 (+ 177us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:52:26.587706 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0811 02:52:26.588164 (+ 458us) server_negotiation.cc:299] Negotiation successful
0811 02:52:26.588270 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260811 02:52:26.589288 21869 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:52:26.574471 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.138.254:43149 (local address 127.12.138.193:50007)
0811 02:52:26.574937 (+ 466us) negotiation.cc:107] Waiting for socket to connect
0811 02:52:26.574991 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0811 02:52:26.575938 (+ 947us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:52:26.576843 (+ 905us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:52:26.576858 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:52:26.577409 (+ 551us) client_negotiation.cc:189] Negotiated authn=SASL
0811 02:52:26.578149 (+ 740us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:52:26.578171 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:52:26.579483 (+ 1312us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:52:26.579489 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:52:26.581048 (+ 1559us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:52:26.581059 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:52:26.581793 (+ 734us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:52:26.581797 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:52:26.581944 (+ 147us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:52:26.582660 (+ 716us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 02:52:26.582684 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0811 02:52:26.584668 (+ 1984us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 02:52:26.586602 (+ 1934us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:52:26.586608 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:52:26.586621 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:52:26.586956 (+ 335us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:52:26.587212 (+ 256us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:52:26.587215 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:52:26.587216 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:52:26.587369 (+ 153us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:52:26.587718 (+ 349us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 02:52:26.587723 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 02:52:26.587957 (+ 234us) client_negotiation.cc:769] Sending connection context
0811 02:52:26.588115 (+ 158us) client_negotiation.cc:240] Negotiation successful
0811 02:52:26.588381 (+ 266us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":261,"thread_start_us":135,"threads_started":1}
I20260811 02:52:26.590354 12843 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDawEwS/build/debug/bin/kudu as pid 21841
I20260811 02:52:26.590508 12843 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786416707705494-12843-0/minicluster-data/ts-0/wal/instance
I20260811 02:52:26.590809 21989 heartbeater.cc:347] Connected to a master server at 127.12.138.254:43149
I20260811 02:52:26.590890 12843 external_mini_cluster.cc:1428] Setting key 927613f1b96001b30fa6d72050237e62
I20260811 02:52:26.590969 21989 heartbeater.cc:464] Registering TS with master...
I20260811 02:52:26.591591 21989 heartbeater.cc:511] Master 127.12.138.254:43149 requested a full tablet report, sending...
I20260811 02:52:26.595093 21761 ts_manager.cc:194] Registered new tserver with Master: 6937bfa1e57344d78f8cc3b5df278e87 (127.12.138.193:34941)
I20260811 02:52:26.596249 21761 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.12.138.193@KRBTEST.COM'} at 127.12.138.193:50007
WARNING: no policy specified for oryx/127.12.138.194@KRBTEST.COM; defaulting to no policy
I20260811 02:52:26.604736 21995 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:52:26.593047 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.138.254:43149 (local address 127.12.138.193:56511)
0811 02:52:26.596628 (+ 3581us) negotiation.cc:107] Waiting for socket to connect
0811 02:52:26.596641 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 02:52:26.596764 (+ 123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:52:26.597091 (+ 327us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:52:26.597095 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:52:26.597383 (+ 288us) client_negotiation.cc:189] Negotiated authn=SASL
0811 02:52:26.597505 (+ 122us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:52:26.597513 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:52:26.598696 (+ 1183us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:52:26.598700 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:52:26.599568 (+ 868us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:52:26.599584 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:52:26.600129 (+ 545us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:52:26.600137 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:52:26.600272 (+ 135us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:52:26.600864 (+ 592us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 02:52:26.600883 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0811 02:52:26.601411 (+ 528us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 02:52:26.603261 (+ 1850us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:52:26.603264 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:52:26.603268 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:52:26.603559 (+ 291us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:52:26.603835 (+ 276us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:52:26.603837 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:52:26.603838 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:52:26.603921 (+ 83us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:52:26.604231 (+ 310us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 02:52:26.604233 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 02:52:26.604333 (+ 100us) client_negotiation.cc:769] Sending connection context
0811 02:52:26.604446 (+ 113us) client_negotiation.cc:240] Negotiation successful
0811 02:52:26.604584 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3486,"thread_start_us":136,"threads_started":1}
I20260811 02:52:26.604874 21816 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:52:26.593132 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.138.193:56511 (local address 127.12.138.254:43149)
0811 02:52:26.593274 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0811 02:52:26.593279 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 02:52:26.596721 (+ 3442us) server_negotiation.cc:407] Connection header received
0811 02:52:26.596929 (+ 208us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:52:26.596932 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:52:26.596978 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:52:26.597073 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:52:26.597644 (+ 571us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:52:26.598535 (+ 891us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:52:26.599733 (+ 1198us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:52:26.599984 (+ 251us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:52:26.600089 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:52:26.601565 (+ 1476us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:52:26.601568 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:52:26.601570 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:52:26.601593 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:52:26.603142 (+ 1549us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:52:26.603661 (+ 519us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:52:26.603663 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:52:26.603664 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:52:26.603747 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:52:26.603985 (+ 238us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:52:26.603987 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:52:26.603988 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:52:26.604153 (+ 165us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:52:26.604225 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0811 02:52:26.604577 (+ 352us) server_negotiation.cc:299] Negotiation successful
0811 02:52:26.604709 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.12.138.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.12.138.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786416707705494-12843-0/krb5kdc/oryx_127.12.138.194.keytab.
Entry for principal oryx/127.12.138.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786416707705494-12843-0/krb5kdc/oryx_127.12.138.194.keytab.
I20260811 02:52:26.626084 12843 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDawEwS/build/debug/bin/kudu
/tmp/dist-test-taskDawEwS/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786416707705494-12843-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786416707705494-12843-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786416707705494-12843-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786416707705494-12843-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.138.194:0
--local_ip_for_outbound_sockets=127.12.138.194
--webserver_interface=127.12.138.194
--webserver_port=0
--tserver_master_addrs=127.12.138.254:43149
--keytab_file=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786416707705494-12843-0/krb5kdc/oryx_127.12.138.194.keytab
--principal=oryx/127.12.138.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.138.212:36165
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786416707705494-12843-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786416707705494-12843-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786416707705494-12843-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 02:52:26.745666 21999 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 02:52:26.745970 21999 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 02:52:26.746076 21999 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 02:52:26.750676 21999 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 02:52:26.750766 21999 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 02:52:26.750851 21999 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260811 02:52:26.750943 21999 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.138.194
I20260811 02:52:26.756819 21999 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.138.212:36165
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786416707705494-12843-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786416707705494-12843-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786416707705494-12843-0/krb5kdc/oryx_127.12.138.194.keytab
--principal=oryx/127.12.138.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.138.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786416707705494-12843-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.12.138.194
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.12.138.254:43149
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21999
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.138.194
--log_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786416707705494-12843-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 02:09:49 UTC on ccycloud.quasar-fsnqqc.root.comops.site
build id 4579570
I20260811 02:52:26.758208 21999 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 02:52:26.759582 21999 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 02:52:26.770537 22004 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:52:26.770537 22005 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:52:26.770543 22007 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:52:26.770908 21999 server_base.cc:1034] running on GCE node
I20260811 02:52:26.771478 21999 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 02:52:26.772265 21999 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 02:52:26.773466 21999 hybrid_clock.cc:648] HybridClock initialized: now 1786416746773450 us; error 37 us; skew 500 ppm
Aug 11 02:52:26 dist-test-slave-9gcw krb5kdc[21711](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416746, etypes {rep=17 tkt=17 ses=17}, oryx/127.12.138.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 02:52:26.776482 21999 init.cc:377] Logged in from keytab as oryx/127.12.138.194@KRBTEST.COM (short username oryx)
I20260811 02:52:26.777552 21999 webserver.cc:460] Webserver started at http://127.12.138.194:45583/ using document root <none> and password file <none>
I20260811 02:52:26.778167 21999 fs_manager.cc:359] Metadata directory not provided
I20260811 02:52:26.778262 21999 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 02:52:26.778514 21999 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 02:52:26.780691 21999 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786416707705494-12843-0/minicluster-data/ts-1/data/instance:
uuid: "6c8a58fdbe944622b27ade12cd92a62e"
format_stamp: "Formatted at 2026-08-11 02:52:26 on dist-test-slave-9gcw"
server_key: "77b43152b76b637a430e6559f4c7e73e"
server_key_iv: "05801b688e11434882d675cb3664da67"
server_key_version: "encryptionkey@0"
I20260811 02:52:26.781373 21999 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786416707705494-12843-0/minicluster-data/ts-1/wal/instance:
uuid: "6c8a58fdbe944622b27ade12cd92a62e"
format_stamp: "Formatted at 2026-08-11 02:52:26 on dist-test-slave-9gcw"
server_key: "77b43152b76b637a430e6559f4c7e73e"
server_key_iv: "05801b688e11434882d675cb3664da67"
server_key_version: "encryptionkey@0"
I20260811 02:52:26.785557 21999 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260811 02:52:26.788142 22014 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 02:52:26.789520 21999 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 02:52:26.789660 21999 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786416707705494-12843-0/minicluster-data/ts-1/data,/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786416707705494-12843-0/minicluster-data/ts-1/wal
uuid: "6c8a58fdbe944622b27ade12cd92a62e"
format_stamp: "Formatted at 2026-08-11 02:52:26 on dist-test-slave-9gcw"
server_key: "77b43152b76b637a430e6559f4c7e73e"
server_key_iv: "05801b688e11434882d675cb3664da67"
server_key_version: "encryptionkey@0"
I20260811 02:52:26.789786 21999 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786416707705494-12843-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786416707705494-12843-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786416707705494-12843-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 02:52:26.802035 21999 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 02:52:26.805505 21999 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 02:52:26.805720 21999 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 02:52:26.807286 21999 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 02:52:26.807363 21999 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 02:52:26.807435 21999 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 02:52:26.807471 21999 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 11 02:52:26 dist-test-slave-9gcw krb5kdc[21711](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416746, etypes {rep=17 tkt=17 ses=17}, oryx/127.12.138.194@KRBTEST.COM for oryx/127.12.138.254@KRBTEST.COM
I20260811 02:52:26.818959 21999 rpc_server.cc:307] RPC server started. Bound to: 127.12.138.194:39121
I20260811 02:52:26.819007 22142 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.138.194:39121 every 8 connection(s)
I20260811 02:52:26.820021 21999 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786416707705494-12843-0/minicluster-data/ts-1/data/info.pb
I20260811 02:52:26.822628 22025 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:52:26.808598 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.138.254:43149 (local address 127.12.138.194:35989)
0811 02:52:26.809072 (+ 474us) negotiation.cc:107] Waiting for socket to connect
0811 02:52:26.809115 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0811 02:52:26.810114 (+ 999us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:52:26.811003 (+ 889us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:52:26.811010 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:52:26.811345 (+ 335us) client_negotiation.cc:189] Negotiated authn=SASL
0811 02:52:26.811882 (+ 537us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:52:26.811896 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:52:26.812914 (+ 1018us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:52:26.812918 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:52:26.814297 (+ 1379us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:52:26.814307 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:52:26.814949 (+ 642us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:52:26.814953 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:52:26.815148 (+ 195us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:52:26.815832 (+ 684us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 02:52:26.815848 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0811 02:52:26.817539 (+ 1691us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 02:52:26.819573 (+ 2034us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:52:26.819579 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:52:26.819596 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:52:26.819925 (+ 329us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:52:26.820256 (+ 331us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:52:26.820259 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:52:26.820261 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:52:26.820460 (+ 199us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:52:26.820949 (+ 489us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 02:52:26.820954 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 02:52:26.821246 (+ 292us) client_negotiation.cc:769] Sending connection context
0811 02:52:26.821449 (+ 203us) client_negotiation.cc:240] Negotiation successful
0811 02:52:26.821754 (+ 305us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":290,"thread_start_us":138,"threads_started":1}
I20260811 02:52:26.822923 21816 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:52:26.808012 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.138.194:35989 (local address 127.12.138.254:43149)
0811 02:52:26.808170 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0811 02:52:26.808174 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 02:52:26.809271 (+ 1097us) server_negotiation.cc:407] Connection header received
0811 02:52:26.810422 (+ 1151us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:52:26.810428 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:52:26.810497 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:52:26.810586 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:52:26.812116 (+ 1530us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:52:26.812766 (+ 650us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:52:26.814430 (+ 1664us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:52:26.814771 (+ 341us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:52:26.814896 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:52:26.817717 (+ 2821us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:52:26.817721 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:52:26.817723 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:52:26.817748 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:52:26.819461 (+ 1713us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:52:26.820037 (+ 576us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:52:26.820039 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:52:26.820041 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:52:26.820165 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:52:26.820585 (+ 420us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:52:26.820589 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:52:26.820591 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:52:26.820851 (+ 260us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:52:26.820959 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0811 02:52:26.822612 (+ 1653us) server_negotiation.cc:299] Negotiation successful
0811 02:52:26.822778 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260811 02:52:26.823858 12843 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDawEwS/build/debug/bin/kudu as pid 21999
I20260811 02:52:26.824023 12843 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786416707705494-12843-0/minicluster-data/ts-1/wal/instance
I20260811 02:52:26.824007 22143 heartbeater.cc:347] Connected to a master server at 127.12.138.254:43149
I20260811 02:52:26.824232 22143 heartbeater.cc:464] Registering TS with master...
I20260811 02:52:26.824407 12843 external_mini_cluster.cc:1428] Setting key 5d9e1b789d41495069244f73deedcd14
I20260811 02:52:26.824833 22143 heartbeater.cc:511] Master 127.12.138.254:43149 requested a full tablet report, sending...
I20260811 02:52:26.826016 21760 ts_manager.cc:194] Registered new tserver with Master: 6c8a58fdbe944622b27ade12cd92a62e (127.12.138.194:39121)
I20260811 02:52:26.826716 21760 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.12.138.194@KRBTEST.COM'} at 127.12.138.194:35989
WARNING: no policy specified for oryx/127.12.138.195@KRBTEST.COM; defaulting to no policy
I20260811 02:52:26.836223 21816 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:52:26.827132 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.138.194:43051 (local address 127.12.138.254:43149)
0811 02:52:26.828956 (+ 1824us) server_negotiation.cc:206] Beginning negotiation
0811 02:52:26.828963 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 02:52:26.828981 (+ 18us) server_negotiation.cc:407] Connection header received
0811 02:52:26.829027 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:52:26.829032 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:52:26.829083 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:52:26.829214 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:52:26.829786 (+ 572us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:52:26.830552 (+ 766us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:52:26.831369 (+ 817us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:52:26.831599 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:52:26.831710 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:52:26.832890 (+ 1180us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:52:26.832894 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:52:26.832897 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:52:26.832921 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:52:26.834455 (+ 1534us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:52:26.834895 (+ 440us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:52:26.834901 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:52:26.834903 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:52:26.834989 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:52:26.835272 (+ 283us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:52:26.835300 (+ 28us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:52:26.835302 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:52:26.835536 (+ 234us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:52:26.835655 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0811 02:52:26.835948 (+ 293us) server_negotiation.cc:299] Negotiation successful
0811 02:52:26.836063 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1726}
I20260811 02:52:26.836236 22151 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:52:26.827138 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.138.254:43149 (local address 127.12.138.194:43051)
0811 02:52:26.827465 (+ 327us) negotiation.cc:107] Waiting for socket to connect
0811 02:52:26.827480 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 02:52:26.827574 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:52:26.829252 (+ 1678us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:52:26.829255 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:52:26.829513 (+ 258us) client_negotiation.cc:189] Negotiated authn=SASL
0811 02:52:26.829653 (+ 140us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:52:26.829662 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:52:26.830691 (+ 1029us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:52:26.830694 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:52:26.831241 (+ 547us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:52:26.831249 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:52:26.831724 (+ 475us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:52:26.831728 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:52:26.831839 (+ 111us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:52:26.832401 (+ 562us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 02:52:26.832413 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0811 02:52:26.832738 (+ 325us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 02:52:26.834569 (+ 1831us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:52:26.834574 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:52:26.834576 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:52:26.834790 (+ 214us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:52:26.835085 (+ 295us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:52:26.835089 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:52:26.835091 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:52:26.835178 (+ 87us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:52:26.835644 (+ 466us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 02:52:26.835647 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 02:52:26.835746 (+ 99us) client_negotiation.cc:769] Sending connection context
0811 02:52:26.835887 (+ 141us) client_negotiation.cc:240] Negotiation successful
0811 02:52:26.836051 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":235,"mutex_wait_us":1,"thread_start_us":88,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.12.138.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.12.138.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786416707705494-12843-0/krb5kdc/oryx_127.12.138.195.keytab.
Entry for principal oryx/127.12.138.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786416707705494-12843-0/krb5kdc/oryx_127.12.138.195.keytab.
I20260811 02:52:26.857096 12843 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDawEwS/build/debug/bin/kudu
/tmp/dist-test-taskDawEwS/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786416707705494-12843-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786416707705494-12843-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786416707705494-12843-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786416707705494-12843-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.138.195:0
--local_ip_for_outbound_sockets=127.12.138.195
--webserver_interface=127.12.138.195
--webserver_port=0
--tserver_master_addrs=127.12.138.254:43149
--keytab_file=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786416707705494-12843-0/krb5kdc/oryx_127.12.138.195.keytab
--principal=oryx/127.12.138.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.138.212:36165
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786416707705494-12843-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786416707705494-12843-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786416707705494-12843-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 02:52:26.978569 22155 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 02:52:26.978861 22155 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 02:52:26.978960 22155 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 02:52:26.983003 22155 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 02:52:26.983076 22155 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 02:52:26.983168 22155 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260811 02:52:26.983268 22155 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.138.195
I20260811 02:52:26.988644 22155 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.138.212:36165
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786416707705494-12843-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786416707705494-12843-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786416707705494-12843-0/krb5kdc/oryx_127.12.138.195.keytab
--principal=oryx/127.12.138.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.138.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786416707705494-12843-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.12.138.195
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.12.138.254:43149
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22155
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.138.195
--log_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786416707705494-12843-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 02:09:49 UTC on ccycloud.quasar-fsnqqc.root.comops.site
build id 4579570
I20260811 02:52:26.989835 22155 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 02:52:26.990913 22155 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 02:52:27.001276 22160 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:52:27.001294 22161 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:52:27.001367 22155 server_base.cc:1034] running on GCE node
W20260811 02:52:27.001302 22164 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:52:27.001945 22155 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 02:52:27.002648 22155 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 02:52:27.003849 22155 hybrid_clock.cc:648] HybridClock initialized: now 1786416747003832 us; error 31 us; skew 500 ppm
Aug 11 02:52:27 dist-test-slave-9gcw krb5kdc[21711](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416747, etypes {rep=17 tkt=17 ses=17}, oryx/127.12.138.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 02:52:27.006847 22155 init.cc:377] Logged in from keytab as oryx/127.12.138.195@KRBTEST.COM (short username oryx)
I20260811 02:52:27.007813 22155 webserver.cc:460] Webserver started at http://127.12.138.195:40245/ using document root <none> and password file <none>
I20260811 02:52:27.008396 22155 fs_manager.cc:359] Metadata directory not provided
I20260811 02:52:27.008459 22155 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 02:52:27.008721 22155 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 02:52:27.010926 22155 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786416707705494-12843-0/minicluster-data/ts-2/data/instance:
uuid: "6a04a1a9c41e4f7fb9064c78468ba498"
format_stamp: "Formatted at 2026-08-11 02:52:27 on dist-test-slave-9gcw"
server_key: "b5a191d56dd0460d356bfe43bdd77910"
server_key_iv: "e7336d52868d79360e2186cae377e3cd"
server_key_version: "encryptionkey@0"
I20260811 02:52:27.011508 22155 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786416707705494-12843-0/minicluster-data/ts-2/wal/instance:
uuid: "6a04a1a9c41e4f7fb9064c78468ba498"
format_stamp: "Formatted at 2026-08-11 02:52:27 on dist-test-slave-9gcw"
server_key: "b5a191d56dd0460d356bfe43bdd77910"
server_key_iv: "e7336d52868d79360e2186cae377e3cd"
server_key_version: "encryptionkey@0"
I20260811 02:52:27.015429 22155 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.002s
I20260811 02:52:27.018021 22171 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 02:52:27.019296 22155 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260811 02:52:27.019438 22155 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786416707705494-12843-0/minicluster-data/ts-2/data,/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786416707705494-12843-0/minicluster-data/ts-2/wal
uuid: "6a04a1a9c41e4f7fb9064c78468ba498"
format_stamp: "Formatted at 2026-08-11 02:52:27 on dist-test-slave-9gcw"
server_key: "b5a191d56dd0460d356bfe43bdd77910"
server_key_iv: "e7336d52868d79360e2186cae377e3cd"
server_key_version: "encryptionkey@0"
I20260811 02:52:27.019574 22155 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786416707705494-12843-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786416707705494-12843-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786416707705494-12843-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 02:52:27.036433 22155 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 02:52:27.039724 22155 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 02:52:27.039973 22155 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 02:52:27.041841 22155 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 02:52:27.041924 22155 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 02:52:27.042001 22155 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 02:52:27.042042 22155 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 11 02:52:27 dist-test-slave-9gcw krb5kdc[21711](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416747, etypes {rep=17 tkt=17 ses=17}, oryx/127.12.138.195@KRBTEST.COM for oryx/127.12.138.254@KRBTEST.COM
I20260811 02:52:27.054139 22155 rpc_server.cc:307] RPC server started. Bound to: 127.12.138.195:40285
I20260811 02:52:27.055294 22155 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786416707705494-12843-0/minicluster-data/ts-2/data/info.pb
I20260811 02:52:27.056152 22302 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.138.195:40285 every 8 connection(s)
I20260811 02:52:27.058758 21816 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:52:27.042631 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.138.195:40231 (local address 127.12.138.254:43149)
0811 02:52:27.042768 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0811 02:52:27.042772 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 02:52:27.043918 (+ 1146us) server_negotiation.cc:407] Connection header received
0811 02:52:27.044986 (+ 1068us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:52:27.044990 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:52:27.045056 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:52:27.045119 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:52:27.046739 (+ 1620us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:52:27.047673 (+ 934us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:52:27.049230 (+ 1557us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:52:27.049592 (+ 362us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:52:27.049705 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:52:27.054823 (+ 5118us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:52:27.054827 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:52:27.054829 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:52:27.054860 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:52:27.056612 (+ 1752us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:52:27.057328 (+ 716us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:52:27.057331 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:52:27.057332 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:52:27.057416 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:52:27.057734 (+ 318us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:52:27.057737 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:52:27.057738 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:52:27.057959 (+ 221us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:52:27.058037 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0811 02:52:27.058523 (+ 486us) server_negotiation.cc:299] Negotiation successful
0811 02:52:27.058633 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260811 02:52:27.059641 22181 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:52:27.043235 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.138.254:43149 (local address 127.12.138.195:40231)
0811 02:52:27.043744 (+ 509us) negotiation.cc:107] Waiting for socket to connect
0811 02:52:27.043796 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0811 02:52:27.044690 (+ 894us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:52:27.045631 (+ 941us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:52:27.045640 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:52:27.046008 (+ 368us) client_negotiation.cc:189] Negotiated authn=SASL
0811 02:52:27.046551 (+ 543us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:52:27.046571 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:52:27.047833 (+ 1262us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:52:27.047837 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:52:27.049017 (+ 1180us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:52:27.049055 (+ 38us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:52:27.049952 (+ 897us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:52:27.049956 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:52:27.050162 (+ 206us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:52:27.051038 (+ 876us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 02:52:27.051060 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0811 02:52:27.054546 (+ 3486us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 02:52:27.056723 (+ 2177us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:52:27.056752 (+ 29us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:52:27.056769 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:52:27.057215 (+ 446us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:52:27.057503 (+ 288us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:52:27.057505 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:52:27.057507 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:52:27.057647 (+ 140us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:52:27.058048 (+ 401us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 02:52:27.058053 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 02:52:27.058289 (+ 236us) client_negotiation.cc:769] Sending connection context
0811 02:52:27.058458 (+ 169us) client_negotiation.cc:240] Negotiation successful
0811 02:52:27.058715 (+ 257us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":300,"thread_start_us":132,"threads_started":1}
I20260811 02:52:27.060843 22303 heartbeater.cc:347] Connected to a master server at 127.12.138.254:43149
I20260811 02:52:27.060998 22303 heartbeater.cc:464] Registering TS with master...
I20260811 02:52:27.061614 22303 heartbeater.cc:511] Master 127.12.138.254:43149 requested a full tablet report, sending...
I20260811 02:52:27.062783 21761 ts_manager.cc:194] Registered new tserver with Master: 6a04a1a9c41e4f7fb9064c78468ba498 (127.12.138.195:40285)
I20260811 02:52:27.063625 21761 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.12.138.195@KRBTEST.COM'} at 127.12.138.195:40231
I20260811 02:52:27.065454 12843 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDawEwS/build/debug/bin/kudu as pid 22155
I20260811 02:52:27.065598 12843 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786416707705494-12843-0/minicluster-data/ts-2/wal/instance
I20260811 02:52:27.065953 12843 external_mini_cluster.cc:1428] Setting key 9f8bbbff47fa6c271f41d46997fd533a
I20260811 02:52:27.068270 12843 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 02:52:27.071434 21816 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:52:27.063375 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.138.195:44197 (local address 127.12.138.254:43149)
0811 02:52:27.063525 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0811 02:52:27.063529 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 02:52:27.063696 (+ 167us) server_negotiation.cc:407] Connection header received
0811 02:52:27.063848 (+ 152us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:52:27.063851 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:52:27.063908 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:52:27.063988 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:52:27.064493 (+ 505us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:52:27.065258 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:52:27.066344 (+ 1086us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:52:27.066582 (+ 238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:52:27.066699 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:52:27.067953 (+ 1254us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:52:27.067957 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:52:27.067960 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:52:27.067984 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:52:27.069640 (+ 1656us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:52:27.070082 (+ 442us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:52:27.070088 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:52:27.070093 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:52:27.070186 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:52:27.070521 (+ 335us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:52:27.070527 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:52:27.070531 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:52:27.070783 (+ 252us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:52:27.070907 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0811 02:52:27.071221 (+ 314us) server_negotiation.cc:299] Negotiation successful
0811 02:52:27.071319 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260811 02:52:27.071605 22310 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:52:27.063268 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.138.254:43149 (local address 127.12.138.195:44197)
0811 02:52:27.063612 (+ 344us) negotiation.cc:107] Waiting for socket to connect
0811 02:52:27.063624 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 02:52:27.063741 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:52:27.064024 (+ 283us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:52:27.064027 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:52:27.064255 (+ 228us) client_negotiation.cc:189] Negotiated authn=SASL
0811 02:52:27.064371 (+ 116us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:52:27.064377 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:52:27.065410 (+ 1033us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:52:27.065414 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:52:27.066235 (+ 821us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:52:27.066247 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:52:27.066700 (+ 453us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:52:27.066703 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:52:27.066814 (+ 111us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:52:27.067438 (+ 624us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 02:52:27.067456 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0811 02:52:27.067824 (+ 368us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 02:52:27.069753 (+ 1929us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:52:27.069756 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:52:27.069758 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:52:27.069983 (+ 225us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:52:27.070313 (+ 330us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:52:27.070316 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:52:27.070318 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:52:27.070416 (+ 98us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:52:27.070922 (+ 506us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 02:52:27.070925 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 02:52:27.071031 (+ 106us) client_negotiation.cc:769] Sending connection context
0811 02:52:27.071163 (+ 132us) client_negotiation.cc:240] Negotiation successful
0811 02:52:27.071338 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":263,"thread_start_us":135,"threads_started":1}
Aug 11 02:52:27 dist-test-slave-9gcw krb5kdc[21711](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416747, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 11 02:52:27 dist-test-slave-9gcw krb5kdc[21711](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.12.138.254@KRBTEST.COM, Server not found in Kerberos database
Aug 11 02:52:27 dist-test-slave-9gcw krb5kdc[21711](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.12.138.254@KRBTEST.COM, Server not found in Kerberos database
W20260811 02:52:27.094973 21816 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 02:52:27.088260 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36664 (local address 127.12.138.254:43149)
0811 02:52:27.088409 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0811 02:52:27.088413 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 02:52:27.088426 (+ 13us) server_negotiation.cc:407] Connection header received
0811 02:52:27.088470 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:52:27.088472 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:52:27.088512 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:52:27.088609 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:52:27.089467 (+ 858us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:52:27.089999 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:52:27.090880 (+ 881us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:52:27.091105 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:52:27.091221 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:52:27.094817 (+ 3596us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:36664: BlockingRecv error: recv got EOF from 127.0.0.1:36664 (error 108)
Metrics: {"server-negotiator.queue_time_us":61}
Aug 11 02:52:27 dist-test-slave-9gcw krb5kdc[21711](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416747, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for oryx/127.12.138.254@KRBTEST.COM
I20260811 02:52:27.104707 21816 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:52:27.096518 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36680 (local address 127.12.138.254:43149)
0811 02:52:27.096636 (+ 118us) server_negotiation.cc:206] Beginning negotiation
0811 02:52:27.096639 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 02:52:27.096805 (+ 166us) server_negotiation.cc:407] Connection header received
0811 02:52:27.096902 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:52:27.096905 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:52:27.096981 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:52:27.097073 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:52:27.097717 (+ 644us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:52:27.098245 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:52:27.099022 (+ 777us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:52:27.099247 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:52:27.099333 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:52:27.101742 (+ 2409us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:52:27.101744 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:52:27.101747 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:52:27.101771 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:52:27.103267 (+ 1496us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:52:27.103669 (+ 402us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:52:27.103671 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:52:27.103673 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:52:27.103748 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:52:27.103983 (+ 235us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:52:27.103985 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:52:27.103986 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:52:27.104155 (+ 169us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:52:27.104255 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0811 02:52:27.104483 (+ 228us) server_negotiation.cc:299] Negotiation successful
0811 02:52:27.104580 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260811 02:52:27.107749 21761 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:36680:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 02:52:27.111200 21761 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 02:52:27.126533 22333 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:52:27.120745 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60306 (local address 127.12.138.195:40285)
0811 02:52:27.121343 (+ 598us) server_negotiation.cc:206] Beginning negotiation
0811 02:52:27.121354 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 02:52:27.121378 (+ 24us) server_negotiation.cc:407] Connection header received
0811 02:52:27.121442 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:52:27.121446 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:52:27.121584 (+ 138us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:52:27.121731 (+ 147us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 02:52:27.122257 (+ 526us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:52:27.123149 (+ 892us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:52:27.124922 (+ 1773us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:52:27.125775 (+ 853us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:52:27.125928 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:52:27.126062 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0811 02:52:27.126265 (+ 203us) server_negotiation.cc:299] Negotiation successful
0811 02:52:27.126365 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":487,"thread_start_us":130,"threads_started":1}
I20260811 02:52:27.126550 22332 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:52:27.120082 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.138.195:40285 (local address 127.0.0.1:60306)
0811 02:52:27.120828 (+ 746us) negotiation.cc:107] Waiting for socket to connect
0811 02:52:27.120842 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 02:52:27.120938 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:52:27.121880 (+ 942us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:52:27.121882 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:52:27.121892 (+ 10us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 02:52:27.122066 (+ 174us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:52:27.122074 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:52:27.123321 (+ 1247us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:52:27.123324 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:52:27.124784 (+ 1460us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:52:27.124796 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:52:27.125900 (+ 1104us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:52:27.125906 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:52:27.126035 (+ 129us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:52:27.126075 (+ 40us) client_negotiation.cc:769] Sending connection context
0811 02:52:27.126221 (+ 146us) client_negotiation.cc:240] Negotiation successful
0811 02:52:27.126360 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":678,"mutex_wait_us":47,"spinlock_wait_cycles":896,"thread_start_us":104,"threads_started":1}
I20260811 02:52:27.126874 22335 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:52:27.120873 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35364 (local address 127.12.138.193:34941)
0811 02:52:27.121235 (+ 362us) server_negotiation.cc:206] Beginning negotiation
0811 02:52:27.121242 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 02:52:27.121269 (+ 27us) server_negotiation.cc:407] Connection header received
0811 02:52:27.121351 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:52:27.121357 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:52:27.121572 (+ 215us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:52:27.121811 (+ 239us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 02:52:27.122301 (+ 490us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:52:27.123467 (+ 1166us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:52:27.124901 (+ 1434us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:52:27.125777 (+ 876us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:52:27.125902 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:52:27.126042 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0811 02:52:27.126634 (+ 592us) server_negotiation.cc:299] Negotiation successful
0811 02:52:27.126745 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":225,"thread_start_us":79,"threads_started":1}
I20260811 02:52:27.126569 22330 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:52:27.120512 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.138.193:34941 (local address 127.0.0.1:35364)
0811 02:52:27.120871 (+ 359us) negotiation.cc:107] Waiting for socket to connect
0811 02:52:27.120885 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 02:52:27.120952 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:52:27.121774 (+ 822us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:52:27.121776 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:52:27.121829 (+ 53us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 02:52:27.122064 (+ 235us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:52:27.122073 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:52:27.123628 (+ 1555us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:52:27.123631 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:52:27.124769 (+ 1138us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:52:27.124779 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:52:27.126110 (+ 1331us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:52:27.126113 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:52:27.126204 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:52:27.126228 (+ 24us) client_negotiation.cc:769] Sending connection context
0811 02:52:27.126310 (+ 82us) client_negotiation.cc:240] Negotiation successful
0811 02:52:27.126386 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":299,"mutex_wait_us":84,"spinlock_wait_cycles":48640,"thread_start_us":101,"threads_started":1}
I20260811 02:52:27.127224 22331 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:52:27.120081 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40316 (local address 127.12.138.194:39121)
0811 02:52:27.121233 (+ 1152us) server_negotiation.cc:206] Beginning negotiation
0811 02:52:27.121245 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 02:52:27.121268 (+ 23us) server_negotiation.cc:407] Connection header received
0811 02:52:27.121346 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:52:27.121353 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:52:27.121584 (+ 231us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:52:27.122000 (+ 416us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 02:52:27.122210 (+ 210us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:52:27.123467 (+ 1257us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:52:27.125423 (+ 1956us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:52:27.126212 (+ 789us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:52:27.126346 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:52:27.126457 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0811 02:52:27.126961 (+ 504us) server_negotiation.cc:299] Negotiation successful
0811 02:52:27.127084 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1015,"thread_start_us":109,"threads_started":1}
I20260811 02:52:27.126894 22329 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:52:27.119916 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.138.194:39121 (local address 127.0.0.1:40316)
0811 02:52:27.120428 (+ 512us) negotiation.cc:107] Waiting for socket to connect
0811 02:52:27.120481 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0811 02:52:27.120618 (+ 137us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:52:27.121767 (+ 1149us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:52:27.121773 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:52:27.121831 (+ 58us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 02:52:27.122064 (+ 233us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:52:27.122073 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:52:27.123582 (+ 1509us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:52:27.123585 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:52:27.125281 (+ 1696us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:52:27.125292 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:52:27.126518 (+ 1226us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:52:27.126522 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:52:27.126618 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:52:27.126640 (+ 22us) client_negotiation.cc:769] Sending connection context
0811 02:52:27.126713 (+ 73us) client_negotiation.cc:240] Negotiation successful
0811 02:52:27.126770 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":332,"mutex_wait_us":95,"spinlock_wait_cycles":6912,"thread_start_us":86,"threads_started":1}
I20260811 02:52:27.130390 22225 tablet_service.cc:1467] Processing CreateTablet for tablet e790f2d8803b4c04953430352b41ba26 (DEFAULT_TABLE table=test-table [id=5ddfba96e43e4cc49085032b56baea05]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 02:52:27.130304 21916 tablet_service.cc:1467] Processing CreateTablet for tablet e790f2d8803b4c04953430352b41ba26 (DEFAULT_TABLE table=test-table [id=5ddfba96e43e4cc49085032b56baea05]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 02:52:27.130303 22073 tablet_service.cc:1467] Processing CreateTablet for tablet e790f2d8803b4c04953430352b41ba26 (DEFAULT_TABLE table=test-table [id=5ddfba96e43e4cc49085032b56baea05]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 02:52:27.131420 22225 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e790f2d8803b4c04953430352b41ba26. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 02:52:27.131433 21916 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e790f2d8803b4c04953430352b41ba26. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 02:52:27.131433 22073 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e790f2d8803b4c04953430352b41ba26. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 02:52:27.138093 22337 tablet_bootstrap.cc:492] T e790f2d8803b4c04953430352b41ba26 P 6a04a1a9c41e4f7fb9064c78468ba498: Bootstrap starting.
I20260811 02:52:27.140080 22338 tablet_bootstrap.cc:492] T e790f2d8803b4c04953430352b41ba26 P 6937bfa1e57344d78f8cc3b5df278e87: Bootstrap starting.
I20260811 02:52:27.140846 22337 tablet_bootstrap.cc:654] T e790f2d8803b4c04953430352b41ba26 P 6a04a1a9c41e4f7fb9064c78468ba498: Neither blocks nor log segments found. Creating new log.
I20260811 02:52:27.141834 22337 log.cc:824] T e790f2d8803b4c04953430352b41ba26 P 6a04a1a9c41e4f7fb9064c78468ba498: Log is configured to *not* fsync() on all Append() calls
I20260811 02:52:27.142371 22338 tablet_bootstrap.cc:654] T e790f2d8803b4c04953430352b41ba26 P 6937bfa1e57344d78f8cc3b5df278e87: Neither blocks nor log segments found. Creating new log.
I20260811 02:52:27.142640 22340 tablet_bootstrap.cc:492] T e790f2d8803b4c04953430352b41ba26 P 6c8a58fdbe944622b27ade12cd92a62e: Bootstrap starting.
I20260811 02:52:27.143235 22338 log.cc:824] T e790f2d8803b4c04953430352b41ba26 P 6937bfa1e57344d78f8cc3b5df278e87: Log is configured to *not* fsync() on all Append() calls
I20260811 02:52:27.144013 22337 tablet_bootstrap.cc:492] T e790f2d8803b4c04953430352b41ba26 P 6a04a1a9c41e4f7fb9064c78468ba498: No bootstrap required, opened a new log
I20260811 02:52:27.144237 22337 ts_tablet_manager.cc:1397] T e790f2d8803b4c04953430352b41ba26 P 6a04a1a9c41e4f7fb9064c78468ba498: Time spent bootstrapping tablet: real 0.006s user 0.001s sys 0.003s
I20260811 02:52:27.144783 22340 tablet_bootstrap.cc:654] T e790f2d8803b4c04953430352b41ba26 P 6c8a58fdbe944622b27ade12cd92a62e: Neither blocks nor log segments found. Creating new log.
I20260811 02:52:27.145424 22338 tablet_bootstrap.cc:492] T e790f2d8803b4c04953430352b41ba26 P 6937bfa1e57344d78f8cc3b5df278e87: No bootstrap required, opened a new log
I20260811 02:52:27.145622 22338 ts_tablet_manager.cc:1397] T e790f2d8803b4c04953430352b41ba26 P 6937bfa1e57344d78f8cc3b5df278e87: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260811 02:52:27.145655 22340 log.cc:824] T e790f2d8803b4c04953430352b41ba26 P 6c8a58fdbe944622b27ade12cd92a62e: Log is configured to *not* fsync() on all Append() calls
I20260811 02:52:27.147794 22340 tablet_bootstrap.cc:492] T e790f2d8803b4c04953430352b41ba26 P 6c8a58fdbe944622b27ade12cd92a62e: No bootstrap required, opened a new log
I20260811 02:52:27.147982 22340 ts_tablet_manager.cc:1397] T e790f2d8803b4c04953430352b41ba26 P 6c8a58fdbe944622b27ade12cd92a62e: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260811 02:52:27.148012 22337 raft_consensus.cc:348] T e790f2d8803b4c04953430352b41ba26 P 6a04a1a9c41e4f7fb9064c78468ba498 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c8a58fdbe944622b27ade12cd92a62e" member_type: VOTER last_known_addr { host: "127.12.138.194" port: 39121 } } peers { permanent_uuid: "6a04a1a9c41e4f7fb9064c78468ba498" member_type: VOTER last_known_addr { host: "127.12.138.195" port: 40285 } } peers { permanent_uuid: "6937bfa1e57344d78f8cc3b5df278e87" member_type: VOTER last_known_addr { host: "127.12.138.193" port: 34941 } }
I20260811 02:52:27.148329 22337 raft_consensus.cc:374] T e790f2d8803b4c04953430352b41ba26 P 6a04a1a9c41e4f7fb9064c78468ba498 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 02:52:27.148384 22337 raft_consensus.cc:729] T e790f2d8803b4c04953430352b41ba26 P 6a04a1a9c41e4f7fb9064c78468ba498 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6a04a1a9c41e4f7fb9064c78468ba498, State: Initialized, Role: FOLLOWER
I20260811 02:52:27.148846 22337 consensus_queue.cc:260] T e790f2d8803b4c04953430352b41ba26 P 6a04a1a9c41e4f7fb9064c78468ba498 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c8a58fdbe944622b27ade12cd92a62e" member_type: VOTER last_known_addr { host: "127.12.138.194" port: 39121 } } peers { permanent_uuid: "6a04a1a9c41e4f7fb9064c78468ba498" member_type: VOTER last_known_addr { host: "127.12.138.195" port: 40285 } } peers { permanent_uuid: "6937bfa1e57344d78f8cc3b5df278e87" member_type: VOTER last_known_addr { host: "127.12.138.193" port: 34941 } }
I20260811 02:52:27.149540 22303 heartbeater.cc:503] Master 127.12.138.254:43149 was elected leader, sending a full tablet report...
I20260811 02:52:27.149854 22337 ts_tablet_manager.cc:1428] T e790f2d8803b4c04953430352b41ba26 P 6a04a1a9c41e4f7fb9064c78468ba498: Time spent starting tablet: real 0.006s user 0.004s sys 0.002s
I20260811 02:52:27.151811 22340 raft_consensus.cc:348] T e790f2d8803b4c04953430352b41ba26 P 6c8a58fdbe944622b27ade12cd92a62e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c8a58fdbe944622b27ade12cd92a62e" member_type: VOTER last_known_addr { host: "127.12.138.194" port: 39121 } } peers { permanent_uuid: "6a04a1a9c41e4f7fb9064c78468ba498" member_type: VOTER last_known_addr { host: "127.12.138.195" port: 40285 } } peers { permanent_uuid: "6937bfa1e57344d78f8cc3b5df278e87" member_type: VOTER last_known_addr { host: "127.12.138.193" port: 34941 } }
I20260811 02:52:27.152087 22340 raft_consensus.cc:374] T e790f2d8803b4c04953430352b41ba26 P 6c8a58fdbe944622b27ade12cd92a62e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 02:52:27.152161 22340 raft_consensus.cc:729] T e790f2d8803b4c04953430352b41ba26 P 6c8a58fdbe944622b27ade12cd92a62e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6c8a58fdbe944622b27ade12cd92a62e, State: Initialized, Role: FOLLOWER
I20260811 02:52:27.152626 22340 consensus_queue.cc:260] T e790f2d8803b4c04953430352b41ba26 P 6c8a58fdbe944622b27ade12cd92a62e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c8a58fdbe944622b27ade12cd92a62e" member_type: VOTER last_known_addr { host: "127.12.138.194" port: 39121 } } peers { permanent_uuid: "6a04a1a9c41e4f7fb9064c78468ba498" member_type: VOTER last_known_addr { host: "127.12.138.195" port: 40285 } } peers { permanent_uuid: "6937bfa1e57344d78f8cc3b5df278e87" member_type: VOTER last_known_addr { host: "127.12.138.193" port: 34941 } }
I20260811 02:52:27.153172 22338 raft_consensus.cc:348] T e790f2d8803b4c04953430352b41ba26 P 6937bfa1e57344d78f8cc3b5df278e87 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c8a58fdbe944622b27ade12cd92a62e" member_type: VOTER last_known_addr { host: "127.12.138.194" port: 39121 } } peers { permanent_uuid: "6a04a1a9c41e4f7fb9064c78468ba498" member_type: VOTER last_known_addr { host: "127.12.138.195" port: 40285 } } peers { permanent_uuid: "6937bfa1e57344d78f8cc3b5df278e87" member_type: VOTER last_known_addr { host: "127.12.138.193" port: 34941 } }
I20260811 02:52:27.153362 22143 heartbeater.cc:503] Master 127.12.138.254:43149 was elected leader, sending a full tablet report...
I20260811 02:52:27.153438 22338 raft_consensus.cc:374] T e790f2d8803b4c04953430352b41ba26 P 6937bfa1e57344d78f8cc3b5df278e87 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 02:52:27.153489 22338 raft_consensus.cc:729] T e790f2d8803b4c04953430352b41ba26 P 6937bfa1e57344d78f8cc3b5df278e87 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6937bfa1e57344d78f8cc3b5df278e87, State: Initialized, Role: FOLLOWER
I20260811 02:52:27.153679 22340 ts_tablet_manager.cc:1428] T e790f2d8803b4c04953430352b41ba26 P 6c8a58fdbe944622b27ade12cd92a62e: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260811 02:52:27.154574 22338 consensus_queue.cc:260] T e790f2d8803b4c04953430352b41ba26 P 6937bfa1e57344d78f8cc3b5df278e87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c8a58fdbe944622b27ade12cd92a62e" member_type: VOTER last_known_addr { host: "127.12.138.194" port: 39121 } } peers { permanent_uuid: "6a04a1a9c41e4f7fb9064c78468ba498" member_type: VOTER last_known_addr { host: "127.12.138.195" port: 40285 } } peers { permanent_uuid: "6937bfa1e57344d78f8cc3b5df278e87" member_type: VOTER last_known_addr { host: "127.12.138.193" port: 34941 } }
I20260811 02:52:27.155207 21989 heartbeater.cc:503] Master 127.12.138.254:43149 was elected leader, sending a full tablet report...
I20260811 02:52:27.156229 22338 ts_tablet_manager.cc:1428] T e790f2d8803b4c04953430352b41ba26 P 6937bfa1e57344d78f8cc3b5df278e87: Time spent starting tablet: real 0.011s user 0.006s sys 0.000s
W20260811 02:52:27.307443 22304 tablet.cc:2369] T e790f2d8803b4c04953430352b41ba26 P 6a04a1a9c41e4f7fb9064c78468ba498: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 02:52:27.321364 22145 tablet.cc:2369] T e790f2d8803b4c04953430352b41ba26 P 6c8a58fdbe944622b27ade12cd92a62e: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 02:52:27.336962 21990 tablet.cc:2369] T e790f2d8803b4c04953430352b41ba26 P 6937bfa1e57344d78f8cc3b5df278e87: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 02:52:27.440662 22345 raft_consensus.cc:482] T e790f2d8803b4c04953430352b41ba26 P 6c8a58fdbe944622b27ade12cd92a62e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 02:52:27.440963 22345 raft_consensus.cc:504] T e790f2d8803b4c04953430352b41ba26 P 6c8a58fdbe944622b27ade12cd92a62e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c8a58fdbe944622b27ade12cd92a62e" member_type: VOTER last_known_addr { host: "127.12.138.194" port: 39121 } } peers { permanent_uuid: "6a04a1a9c41e4f7fb9064c78468ba498" member_type: VOTER last_known_addr { host: "127.12.138.195" port: 40285 } } peers { permanent_uuid: "6937bfa1e57344d78f8cc3b5df278e87" member_type: VOTER last_known_addr { host: "127.12.138.193" port: 34941 } }
I20260811 02:52:27.442241 22345 leader_election.cc:288] T e790f2d8803b4c04953430352b41ba26 P 6c8a58fdbe944622b27ade12cd92a62e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6a04a1a9c41e4f7fb9064c78468ba498 (127.12.138.195:40285), 6937bfa1e57344d78f8cc3b5df278e87 (127.12.138.193:34941)
I20260811 02:52:27.446487 22347 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:52:27.442547 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.138.195:40285 (local address 127.12.138.194:34203)
0811 02:52:27.443074 (+ 527us) negotiation.cc:107] Waiting for socket to connect
0811 02:52:27.443085 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0811 02:52:27.443224 (+ 139us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:52:27.443507 (+ 283us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:52:27.443510 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:52:27.443551 (+ 41us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 02:52:27.443663 (+ 112us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:52:27.443668 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:52:27.444423 (+ 755us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:52:27.444428 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:52:27.445374 (+ 946us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:52:27.445384 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:52:27.446034 (+ 650us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:52:27.446038 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:52:27.446136 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:52:27.446162 (+ 26us) client_negotiation.cc:769] Sending connection context
0811 02:52:27.446242 (+ 80us) client_negotiation.cc:240] Negotiation successful
0811 02:52:27.446317 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":469,"mutex_wait_us":144,"spinlock_wait_cycles":4224,"thread_start_us":65,"threads_started":1}
I20260811 02:52:27.446484 22333 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:52:27.442730 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.138.194:34203 (local address 127.12.138.195:40285)
0811 02:52:27.442895 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0811 02:52:27.442900 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 02:52:27.443187 (+ 287us) server_negotiation.cc:407] Connection header received
0811 02:52:27.443367 (+ 180us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:52:27.443370 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:52:27.443410 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:52:27.443498 (+ 88us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 02:52:27.443785 (+ 287us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:52:27.444306 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:52:27.445502 (+ 1196us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:52:27.445895 (+ 393us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:52:27.446033 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:52:27.446091 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0811 02:52:27.446269 (+ 178us) server_negotiation.cc:299] Negotiation successful
0811 02:52:27.446359 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260811 02:52:27.446532 22335 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:52:27.442803 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.138.194:59797 (local address 127.12.138.193:34941)
0811 02:52:27.442936 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0811 02:52:27.442940 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 02:52:27.443122 (+ 182us) server_negotiation.cc:407] Connection header received
0811 02:52:27.443309 (+ 187us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:52:27.443312 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:52:27.443364 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:52:27.443462 (+ 98us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 02:52:27.443762 (+ 300us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:52:27.444303 (+ 541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:52:27.445484 (+ 1181us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:52:27.445915 (+ 431us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:52:27.446080 (+ 165us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:52:27.446133 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0811 02:52:27.446348 (+ 215us) server_negotiation.cc:299] Negotiation successful
0811 02:52:27.446423 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260811 02:52:27.446967 22348 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:52:27.442547 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.138.193:34941 (local address 127.12.138.194:59797)
0811 02:52:27.443010 (+ 463us) negotiation.cc:107] Waiting for socket to connect
0811 02:52:27.443024 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 02:52:27.443177 (+ 153us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:52:27.443475 (+ 298us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:52:27.443478 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:52:27.443494 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 02:52:27.443643 (+ 149us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:52:27.443650 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:52:27.444429 (+ 779us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:52:27.444435 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:52:27.445352 (+ 917us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:52:27.445363 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:52:27.446075 (+ 712us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:52:27.446079 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:52:27.446182 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:52:27.446204 (+ 22us) client_negotiation.cc:769] Sending connection context
0811 02:52:27.446722 (+ 518us) client_negotiation.cc:240] Negotiation successful
0811 02:52:27.446805 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":389,"mutex_wait_us":67,"spinlock_wait_cycles":16768,"thread_start_us":94,"threads_started":1}
I20260811 02:52:27.447331 22249 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e790f2d8803b4c04953430352b41ba26" candidate_uuid: "6c8a58fdbe944622b27ade12cd92a62e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6a04a1a9c41e4f7fb9064c78468ba498" is_pre_election: true
I20260811 02:52:27.447623 22249 raft_consensus.cc:2454] T e790f2d8803b4c04953430352b41ba26 P 6a04a1a9c41e4f7fb9064c78468ba498 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c8a58fdbe944622b27ade12cd92a62e in term 0.
I20260811 02:52:27.447603 21937 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e790f2d8803b4c04953430352b41ba26" candidate_uuid: "6c8a58fdbe944622b27ade12cd92a62e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6937bfa1e57344d78f8cc3b5df278e87" is_pre_election: true
I20260811 02:52:27.447839 21937 raft_consensus.cc:2454] T e790f2d8803b4c04953430352b41ba26 P 6937bfa1e57344d78f8cc3b5df278e87 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c8a58fdbe944622b27ade12cd92a62e in term 0.
I20260811 02:52:27.448168 22016 leader_election.cc:302] T e790f2d8803b4c04953430352b41ba26 P 6c8a58fdbe944622b27ade12cd92a62e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a04a1a9c41e4f7fb9064c78468ba498, 6c8a58fdbe944622b27ade12cd92a62e; no voters:
I20260811 02:52:27.448516 22345 raft_consensus.cc:2781] T e790f2d8803b4c04953430352b41ba26 P 6c8a58fdbe944622b27ade12cd92a62e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 02:52:27.448598 22345 raft_consensus.cc:482] T e790f2d8803b4c04953430352b41ba26 P 6c8a58fdbe944622b27ade12cd92a62e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 02:52:27.448652 22345 raft_consensus.cc:3037] T e790f2d8803b4c04953430352b41ba26 P 6c8a58fdbe944622b27ade12cd92a62e [term 0 FOLLOWER]: Advancing to term 1
I20260811 02:52:27.449771 22345 raft_consensus.cc:504] T e790f2d8803b4c04953430352b41ba26 P 6c8a58fdbe944622b27ade12cd92a62e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c8a58fdbe944622b27ade12cd92a62e" member_type: VOTER last_known_addr { host: "127.12.138.194" port: 39121 } } peers { permanent_uuid: "6a04a1a9c41e4f7fb9064c78468ba498" member_type: VOTER last_known_addr { host: "127.12.138.195" port: 40285 } } peers { permanent_uuid: "6937bfa1e57344d78f8cc3b5df278e87" member_type: VOTER last_known_addr { host: "127.12.138.193" port: 34941 } }
I20260811 02:52:27.450166 22345 leader_election.cc:288] T e790f2d8803b4c04953430352b41ba26 P 6c8a58fdbe944622b27ade12cd92a62e [CANDIDATE]: Term 1 election: Requested vote from peers 6a04a1a9c41e4f7fb9064c78468ba498 (127.12.138.195:40285), 6937bfa1e57344d78f8cc3b5df278e87 (127.12.138.193:34941)
I20260811 02:52:27.450570 21937 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e790f2d8803b4c04953430352b41ba26" candidate_uuid: "6c8a58fdbe944622b27ade12cd92a62e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6937bfa1e57344d78f8cc3b5df278e87"
I20260811 02:52:27.450630 22249 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e790f2d8803b4c04953430352b41ba26" candidate_uuid: "6c8a58fdbe944622b27ade12cd92a62e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6a04a1a9c41e4f7fb9064c78468ba498"
I20260811 02:52:27.450748 21937 raft_consensus.cc:3037] T e790f2d8803b4c04953430352b41ba26 P 6937bfa1e57344d78f8cc3b5df278e87 [term 0 FOLLOWER]: Advancing to term 1
I20260811 02:52:27.450894 22249 raft_consensus.cc:3037] T e790f2d8803b4c04953430352b41ba26 P 6a04a1a9c41e4f7fb9064c78468ba498 [term 0 FOLLOWER]: Advancing to term 1
I20260811 02:52:27.451902 21937 raft_consensus.cc:2454] T e790f2d8803b4c04953430352b41ba26 P 6937bfa1e57344d78f8cc3b5df278e87 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c8a58fdbe944622b27ade12cd92a62e in term 1.
I20260811 02:52:27.452306 22018 leader_election.cc:302] T e790f2d8803b4c04953430352b41ba26 P 6c8a58fdbe944622b27ade12cd92a62e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6937bfa1e57344d78f8cc3b5df278e87, 6c8a58fdbe944622b27ade12cd92a62e; no voters:
I20260811 02:52:27.452497 22345 raft_consensus.cc:2781] T e790f2d8803b4c04953430352b41ba26 P 6c8a58fdbe944622b27ade12cd92a62e [term 1 FOLLOWER]: Leader election won for term 1
I20260811 02:52:27.452607 22249 raft_consensus.cc:2454] T e790f2d8803b4c04953430352b41ba26 P 6a04a1a9c41e4f7fb9064c78468ba498 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c8a58fdbe944622b27ade12cd92a62e in term 1.
I20260811 02:52:27.452695 22345 raft_consensus.cc:686] T e790f2d8803b4c04953430352b41ba26 P 6c8a58fdbe944622b27ade12cd92a62e [term 1 LEADER]: Becoming Leader. State: Replica: 6c8a58fdbe944622b27ade12cd92a62e, State: Running, Role: LEADER
I20260811 02:52:27.453086 22345 consensus_queue.cc:237] T e790f2d8803b4c04953430352b41ba26 P 6c8a58fdbe944622b27ade12cd92a62e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c8a58fdbe944622b27ade12cd92a62e" member_type: VOTER last_known_addr { host: "127.12.138.194" port: 39121 } } peers { permanent_uuid: "6a04a1a9c41e4f7fb9064c78468ba498" member_type: VOTER last_known_addr { host: "127.12.138.195" port: 40285 } } peers { permanent_uuid: "6937bfa1e57344d78f8cc3b5df278e87" member_type: VOTER last_known_addr { host: "127.12.138.193" port: 34941 } }
I20260811 02:52:27.456499 21761 catalog_manager.cc:5495] T e790f2d8803b4c04953430352b41ba26 P 6c8a58fdbe944622b27ade12cd92a62e reported cstate change: term changed from 0 to 1, leader changed from <none> to 6c8a58fdbe944622b27ade12cd92a62e (127.12.138.194). New cstate: current_term: 1 leader_uuid: "6c8a58fdbe944622b27ade12cd92a62e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c8a58fdbe944622b27ade12cd92a62e" member_type: VOTER last_known_addr { host: "127.12.138.194" port: 39121 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6a04a1a9c41e4f7fb9064c78468ba498" member_type: VOTER last_known_addr { host: "127.12.138.195" port: 40285 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6937bfa1e57344d78f8cc3b5df278e87" member_type: VOTER last_known_addr { host: "127.12.138.193" port: 34941 } health_report { overall_health: UNKNOWN } } }
I20260811 02:52:27.499449 22331 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:52:27.495952 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40330 (local address 127.12.138.194:39121)
0811 02:52:27.496082 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0811 02:52:27.496086 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 02:52:27.496100 (+ 14us) server_negotiation.cc:407] Connection header received
0811 02:52:27.496254 (+ 154us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:52:27.496256 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:52:27.496329 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:52:27.496424 (+ 95us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 02:52:27.496685 (+ 261us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:52:27.497212 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:52:27.498058 (+ 846us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:52:27.498263 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:52:27.498351 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:52:27.498601 (+ 250us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 02:52:27.499054 (+ 453us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 02:52:27.499144 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0811 02:52:27.499213 (+ 69us) server_negotiation.cc:299] Negotiation successful
0811 02:52:27.499262 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260811 02:52:27.505398 21937 raft_consensus.cc:1261] T e790f2d8803b4c04953430352b41ba26 P 6937bfa1e57344d78f8cc3b5df278e87 [term 1 FOLLOWER]: Refusing update from remote peer 6c8a58fdbe944622b27ade12cd92a62e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 02:52:27.505398 22249 raft_consensus.cc:1261] T e790f2d8803b4c04953430352b41ba26 P 6a04a1a9c41e4f7fb9064c78468ba498 [term 1 FOLLOWER]: Refusing update from remote peer 6c8a58fdbe944622b27ade12cd92a62e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 02:52:27.506090 22345 consensus_queue.cc:1035] T e790f2d8803b4c04953430352b41ba26 P 6c8a58fdbe944622b27ade12cd92a62e [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a04a1a9c41e4f7fb9064c78468ba498" member_type: VOTER last_known_addr { host: "127.12.138.195" port: 40285 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 02:52:27.506340 22351 consensus_queue.cc:1035] T e790f2d8803b4c04953430352b41ba26 P 6c8a58fdbe944622b27ade12cd92a62e [LEADER]: Connected to new peer: Peer: permanent_uuid: "6937bfa1e57344d78f8cc3b5df278e87" member_type: VOTER last_known_addr { host: "127.12.138.193" port: 34941 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 02:52:27.514473 22356 mvcc.cc:205] Tried to move back new op lower bound from 7317162997774868480 to 7317162997570838528. Current Snapshot: MvccSnapshot[applied={T|T < 7317162997774868480}]
I20260811 02:52:27.514768 22355 mvcc.cc:205] Tried to move back new op lower bound from 7317162997774868480 to 7317162997570838528. Current Snapshot: MvccSnapshot[applied={T|T < 7317162997774868480}]
I20260811 02:52:27.521552 21761 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:36680:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 02:52:27.521744 21761 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:36680:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 02:52:27.524479 21761 catalog_manager.cc:5782] T 00000000000000000000000000000000 P acf0fa3758a14749a4a0ec3e106b3dd5: Sending DeleteTablet for 3 replicas of tablet e790f2d8803b4c04953430352b41ba26
I20260811 02:52:27.525279 22073 tablet_service.cc:1514] Processing DeleteTablet for tablet e790f2d8803b4c04953430352b41ba26 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 02:52:27 UTC) from {username='oryx', principal='oryx/127.12.138.254@KRBTEST.COM'} at 127.0.0.1:40316
I20260811 02:52:27.525468 22225 tablet_service.cc:1514] Processing DeleteTablet for tablet e790f2d8803b4c04953430352b41ba26 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 02:52:27 UTC) from {username='oryx', principal='oryx/127.12.138.254@KRBTEST.COM'} at 127.0.0.1:60306
I20260811 02:52:27.525584 21916 tablet_service.cc:1514] Processing DeleteTablet for tablet e790f2d8803b4c04953430352b41ba26 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 02:52:27 UTC) from {username='oryx', principal='oryx/127.12.138.254@KRBTEST.COM'} at 127.0.0.1:35364
I20260811 02:52:27.526319 22366 tablet_replica.cc:330] T e790f2d8803b4c04953430352b41ba26 P 6937bfa1e57344d78f8cc3b5df278e87: stopping tablet replica
I20260811 02:52:27.526324 22364 tablet_replica.cc:330] T e790f2d8803b4c04953430352b41ba26 P 6c8a58fdbe944622b27ade12cd92a62e: stopping tablet replica
I20260811 02:52:27.526636 22364 raft_consensus.cc:2229] T e790f2d8803b4c04953430352b41ba26 P 6c8a58fdbe944622b27ade12cd92a62e [term 1 LEADER]: Raft consensus shutting down.
I20260811 02:52:27.526608 21761 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:36680:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 02:52:27.526676 22366 raft_consensus.cc:2229] T e790f2d8803b4c04953430352b41ba26 P 6937bfa1e57344d78f8cc3b5df278e87 [term 1 FOLLOWER]: Raft consensus shutting down.
W20260811 02:52:27.527022 21761 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 02:52:27.527052 22364 raft_consensus.cc:2258] T e790f2d8803b4c04953430352b41ba26 P 6c8a58fdbe944622b27ade12cd92a62e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 02:52:27.527344 22366 raft_consensus.cc:2258] T e790f2d8803b4c04953430352b41ba26 P 6937bfa1e57344d78f8cc3b5df278e87 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 02:52:27.527932 22365 tablet_replica.cc:330] T e790f2d8803b4c04953430352b41ba26 P 6a04a1a9c41e4f7fb9064c78468ba498: stopping tablet replica
I20260811 02:52:27.528210 22365 raft_consensus.cc:2229] T e790f2d8803b4c04953430352b41ba26 P 6a04a1a9c41e4f7fb9064c78468ba498 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 02:52:27.528502 22365 raft_consensus.cc:2258] T e790f2d8803b4c04953430352b41ba26 P 6a04a1a9c41e4f7fb9064c78468ba498 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 02:52:27.529251 22366 ts_tablet_manager.cc:1905] T e790f2d8803b4c04953430352b41ba26 P 6937bfa1e57344d78f8cc3b5df278e87: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 02:52:27.530027 22365 ts_tablet_manager.cc:1905] T e790f2d8803b4c04953430352b41ba26 P 6a04a1a9c41e4f7fb9064c78468ba498: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 02:52:27.531090 22364 ts_tablet_manager.cc:1905] T e790f2d8803b4c04953430352b41ba26 P 6c8a58fdbe944622b27ade12cd92a62e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 02:52:27.532624 22366 ts_tablet_manager.cc:1918] T e790f2d8803b4c04953430352b41ba26 P 6937bfa1e57344d78f8cc3b5df278e87: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 02:52:27.532644 22073 tablet_service.cc:1467] Processing CreateTablet for tablet a31cde49c96b46f5b5716dcda2590837 (DEFAULT_TABLE table=test-table [id=6accdb9d6e434086808afd669a191c62]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 02:52:27.532740 22365 ts_tablet_manager.cc:1918] T e790f2d8803b4c04953430352b41ba26 P 6a04a1a9c41e4f7fb9064c78468ba498: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 02:52:27.532863 22365 log.cc:1196] T e790f2d8803b4c04953430352b41ba26 P 6a04a1a9c41e4f7fb9064c78468ba498: Deleting WAL directory at /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786416707705494-12843-0/minicluster-data/ts-2/wal/wals/e790f2d8803b4c04953430352b41ba26
I20260811 02:52:27.532884 22366 log.cc:1196] T e790f2d8803b4c04953430352b41ba26 P 6937bfa1e57344d78f8cc3b5df278e87: Deleting WAL directory at /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786416707705494-12843-0/minicluster-data/ts-0/wal/wals/e790f2d8803b4c04953430352b41ba26
I20260811 02:52:27.532972 22073 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a31cde49c96b46f5b5716dcda2590837. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 02:52:27.533250 22365 ts_tablet_manager.cc:1939] T e790f2d8803b4c04953430352b41ba26 P 6a04a1a9c41e4f7fb9064c78468ba498: Deleting consensus metadata
I20260811 02:52:27.533476 22366 ts_tablet_manager.cc:1939] T e790f2d8803b4c04953430352b41ba26 P 6937bfa1e57344d78f8cc3b5df278e87: Deleting consensus metadata
I20260811 02:52:27.533465 22225 tablet_service.cc:1467] Processing CreateTablet for tablet a31cde49c96b46f5b5716dcda2590837 (DEFAULT_TABLE table=test-table [id=6accdb9d6e434086808afd669a191c62]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 02:52:27.533751 22225 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a31cde49c96b46f5b5716dcda2590837. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 02:52:27.534149 22364 ts_tablet_manager.cc:1918] T e790f2d8803b4c04953430352b41ba26 P 6c8a58fdbe944622b27ade12cd92a62e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 02:52:27.534248 22364 log.cc:1196] T e790f2d8803b4c04953430352b41ba26 P 6c8a58fdbe944622b27ade12cd92a62e: Deleting WAL directory at /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786416707705494-12843-0/minicluster-data/ts-1/wal/wals/e790f2d8803b4c04953430352b41ba26
I20260811 02:52:27.534574 22364 ts_tablet_manager.cc:1939] T e790f2d8803b4c04953430352b41ba26 P 6c8a58fdbe944622b27ade12cd92a62e: Deleting consensus metadata
I20260811 02:52:27.535540 21746 catalog_manager.cc:4841] TS 6c8a58fdbe944622b27ade12cd92a62e (127.12.138.194:39121): tablet e790f2d8803b4c04953430352b41ba26 (table test-table [id=5ddfba96e43e4cc49085032b56baea05]) successfully deleted
I20260811 02:52:27.535851 22337 tablet_bootstrap.cc:492] T a31cde49c96b46f5b5716dcda2590837 P 6a04a1a9c41e4f7fb9064c78468ba498: Bootstrap starting.
I20260811 02:52:27.536026 21744 catalog_manager.cc:4841] TS 6a04a1a9c41e4f7fb9064c78468ba498 (127.12.138.195:40285): tablet e790f2d8803b4c04953430352b41ba26 (table test-table [id=5ddfba96e43e4cc49085032b56baea05]) successfully deleted
I20260811 02:52:27.536177 21745 catalog_manager.cc:4841] TS 6937bfa1e57344d78f8cc3b5df278e87 (127.12.138.193:34941): tablet e790f2d8803b4c04953430352b41ba26 (table test-table [id=5ddfba96e43e4cc49085032b56baea05]) successfully deleted
I20260811 02:52:27.536785 21916 tablet_service.cc:1467] Processing CreateTablet for tablet a31cde49c96b46f5b5716dcda2590837 (DEFAULT_TABLE table=test-table [id=6accdb9d6e434086808afd669a191c62]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 02:52:27.536881 22337 tablet_bootstrap.cc:654] T a31cde49c96b46f5b5716dcda2590837 P 6a04a1a9c41e4f7fb9064c78468ba498: Neither blocks nor log segments found. Creating new log.
I20260811 02:52:27.537031 21916 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a31cde49c96b46f5b5716dcda2590837. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 02:52:27.538079 22337 tablet_bootstrap.cc:492] T a31cde49c96b46f5b5716dcda2590837 P 6a04a1a9c41e4f7fb9064c78468ba498: No bootstrap required, opened a new log
I20260811 02:52:27.538188 22337 ts_tablet_manager.cc:1397] T a31cde49c96b46f5b5716dcda2590837 P 6a04a1a9c41e4f7fb9064c78468ba498: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260811 02:52:27.538576 22337 raft_consensus.cc:348] T a31cde49c96b46f5b5716dcda2590837 P 6a04a1a9c41e4f7fb9064c78468ba498 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6937bfa1e57344d78f8cc3b5df278e87" member_type: VOTER last_known_addr { host: "127.12.138.193" port: 34941 } } peers { permanent_uuid: "6c8a58fdbe944622b27ade12cd92a62e" member_type: VOTER last_known_addr { host: "127.12.138.194" port: 39121 } } peers { permanent_uuid: "6a04a1a9c41e4f7fb9064c78468ba498" member_type: VOTER last_known_addr { host: "127.12.138.195" port: 40285 } }
I20260811 02:52:27.538693 22337 raft_consensus.cc:374] T a31cde49c96b46f5b5716dcda2590837 P 6a04a1a9c41e4f7fb9064c78468ba498 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 02:52:27.538733 22337 raft_consensus.cc:729] T a31cde49c96b46f5b5716dcda2590837 P 6a04a1a9c41e4f7fb9064c78468ba498 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6a04a1a9c41e4f7fb9064c78468ba498, State: Initialized, Role: FOLLOWER
I20260811 02:52:27.538877 22340 tablet_bootstrap.cc:492] T a31cde49c96b46f5b5716dcda2590837 P 6c8a58fdbe944622b27ade12cd92a62e: Bootstrap starting.
I20260811 02:52:27.538872 22337 consensus_queue.cc:260] T a31cde49c96b46f5b5716dcda2590837 P 6a04a1a9c41e4f7fb9064c78468ba498 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6937bfa1e57344d78f8cc3b5df278e87" member_type: VOTER last_known_addr { host: "127.12.138.193" port: 34941 } } peers { permanent_uuid: "6c8a58fdbe944622b27ade12cd92a62e" member_type: VOTER last_known_addr { host: "127.12.138.194" port: 39121 } } peers { permanent_uuid: "6a04a1a9c41e4f7fb9064c78468ba498" member_type: VOTER last_known_addr { host: "127.12.138.195" port: 40285 } }
I20260811 02:52:27.539156 22337 ts_tablet_manager.cc:1428] T a31cde49c96b46f5b5716dcda2590837 P 6a04a1a9c41e4f7fb9064c78468ba498: Time spent starting tablet: real 0.001s user 0.001s sys 0.001s
I20260811 02:52:27.539495 22338 tablet_bootstrap.cc:492] T a31cde49c96b46f5b5716dcda2590837 P 6937bfa1e57344d78f8cc3b5df278e87: Bootstrap starting.
I20260811 02:52:27.539876 22340 tablet_bootstrap.cc:654] T a31cde49c96b46f5b5716dcda2590837 P 6c8a58fdbe944622b27ade12cd92a62e: Neither blocks nor log segments found. Creating new log.
I20260811 02:52:27.540524 22338 tablet_bootstrap.cc:654] T a31cde49c96b46f5b5716dcda2590837 P 6937bfa1e57344d78f8cc3b5df278e87: Neither blocks nor log segments found. Creating new log.
I20260811 02:52:27.541350 22340 tablet_bootstrap.cc:492] T a31cde49c96b46f5b5716dcda2590837 P 6c8a58fdbe944622b27ade12cd92a62e: No bootstrap required, opened a new log
I20260811 02:52:27.541446 22340 ts_tablet_manager.cc:1397] T a31cde49c96b46f5b5716dcda2590837 P 6c8a58fdbe944622b27ade12cd92a62e: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260811 02:52:27.541904 22340 raft_consensus.cc:348] T a31cde49c96b46f5b5716dcda2590837 P 6c8a58fdbe944622b27ade12cd92a62e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6937bfa1e57344d78f8cc3b5df278e87" member_type: VOTER last_known_addr { host: "127.12.138.193" port: 34941 } } peers { permanent_uuid: "6c8a58fdbe944622b27ade12cd92a62e" member_type: VOTER last_known_addr { host: "127.12.138.194" port: 39121 } } peers { permanent_uuid: "6a04a1a9c41e4f7fb9064c78468ba498" member_type: VOTER last_known_addr { host: "127.12.138.195" port: 40285 } }
I20260811 02:52:27.542030 22340 raft_consensus.cc:374] T a31cde49c96b46f5b5716dcda2590837 P 6c8a58fdbe944622b27ade12cd92a62e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 02:52:27.542080 22340 raft_consensus.cc:729] T a31cde49c96b46f5b5716dcda2590837 P 6c8a58fdbe944622b27ade12cd92a62e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6c8a58fdbe944622b27ade12cd92a62e, State: Initialized, Role: FOLLOWER
I20260811 02:52:27.542212 22340 consensus_queue.cc:260] T a31cde49c96b46f5b5716dcda2590837 P 6c8a58fdbe944622b27ade12cd92a62e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6937bfa1e57344d78f8cc3b5df278e87" member_type: VOTER last_known_addr { host: "127.12.138.193" port: 34941 } } peers { permanent_uuid: "6c8a58fdbe944622b27ade12cd92a62e" member_type: VOTER last_known_addr { host: "127.12.138.194" port: 39121 } } peers { permanent_uuid: "6a04a1a9c41e4f7fb9064c78468ba498" member_type: VOTER last_known_addr { host: "127.12.138.195" port: 40285 } }
I20260811 02:52:27.542798 22338 tablet_bootstrap.cc:492] T a31cde49c96b46f5b5716dcda2590837 P 6937bfa1e57344d78f8cc3b5df278e87: No bootstrap required, opened a new log
I20260811 02:52:27.542909 22338 ts_tablet_manager.cc:1397] T a31cde49c96b46f5b5716dcda2590837 P 6937bfa1e57344d78f8cc3b5df278e87: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260811 02:52:27.543421 22338 raft_consensus.cc:348] T a31cde49c96b46f5b5716dcda2590837 P 6937bfa1e57344d78f8cc3b5df278e87 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6937bfa1e57344d78f8cc3b5df278e87" member_type: VOTER last_known_addr { host: "127.12.138.193" port: 34941 } } peers { permanent_uuid: "6c8a58fdbe944622b27ade12cd92a62e" member_type: VOTER last_known_addr { host: "127.12.138.194" port: 39121 } } peers { permanent_uuid: "6a04a1a9c41e4f7fb9064c78468ba498" member_type: VOTER last_known_addr { host: "127.12.138.195" port: 40285 } }
I20260811 02:52:27.543546 22338 raft_consensus.cc:374] T a31cde49c96b46f5b5716dcda2590837 P 6937bfa1e57344d78f8cc3b5df278e87 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 02:52:27.543593 22338 raft_consensus.cc:729] T a31cde49c96b46f5b5716dcda2590837 P 6937bfa1e57344d78f8cc3b5df278e87 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6937bfa1e57344d78f8cc3b5df278e87, State: Initialized, Role: FOLLOWER
I20260811 02:52:27.543728 22338 consensus_queue.cc:260] T a31cde49c96b46f5b5716dcda2590837 P 6937bfa1e57344d78f8cc3b5df278e87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6937bfa1e57344d78f8cc3b5df278e87" member_type: VOTER last_known_addr { host: "127.12.138.193" port: 34941 } } peers { permanent_uuid: "6c8a58fdbe944622b27ade12cd92a62e" member_type: VOTER last_known_addr { host: "127.12.138.194" port: 39121 } } peers { permanent_uuid: "6a04a1a9c41e4f7fb9064c78468ba498" member_type: VOTER last_known_addr { host: "127.12.138.195" port: 40285 } }
I20260811 02:52:27.544034 22338 ts_tablet_manager.cc:1428] T a31cde49c96b46f5b5716dcda2590837 P 6937bfa1e57344d78f8cc3b5df278e87: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260811 02:52:27.544489 22340 ts_tablet_manager.cc:1428] T a31cde49c96b46f5b5716dcda2590837 P 6c8a58fdbe944622b27ade12cd92a62e: Time spent starting tablet: real 0.003s user 0.001s sys 0.000s
I20260811 02:52:27.669230 22346 raft_consensus.cc:482] T a31cde49c96b46f5b5716dcda2590837 P 6937bfa1e57344d78f8cc3b5df278e87 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 02:52:27.669410 22346 raft_consensus.cc:504] T a31cde49c96b46f5b5716dcda2590837 P 6937bfa1e57344d78f8cc3b5df278e87 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6937bfa1e57344d78f8cc3b5df278e87" member_type: VOTER last_known_addr { host: "127.12.138.193" port: 34941 } } peers { permanent_uuid: "6c8a58fdbe944622b27ade12cd92a62e" member_type: VOTER last_known_addr { host: "127.12.138.194" port: 39121 } } peers { permanent_uuid: "6a04a1a9c41e4f7fb9064c78468ba498" member_type: VOTER last_known_addr { host: "127.12.138.195" port: 40285 } }
I20260811 02:52:27.670658 22346 leader_election.cc:288] T a31cde49c96b46f5b5716dcda2590837 P 6937bfa1e57344d78f8cc3b5df278e87 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6c8a58fdbe944622b27ade12cd92a62e (127.12.138.194:39121), 6a04a1a9c41e4f7fb9064c78468ba498 (127.12.138.195:40285)
I20260811 02:52:27.675705 22331 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:52:27.671158 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.138.193:45391 (local address 127.12.138.194:39121)
0811 02:52:27.671306 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0811 02:52:27.671310 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 02:52:27.671445 (+ 135us) server_negotiation.cc:407] Connection header received
0811 02:52:27.671613 (+ 168us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:52:27.671616 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:52:27.671665 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:52:27.671759 (+ 94us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 02:52:27.672290 (+ 531us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:52:27.672953 (+ 663us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:52:27.674443 (+ 1490us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:52:27.675021 (+ 578us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:52:27.675146 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:52:27.675206 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0811 02:52:27.675466 (+ 260us) server_negotiation.cc:299] Negotiation successful
0811 02:52:27.675570 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260811 02:52:27.675841 22333 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:52:27.671056 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.138.193:33107 (local address 127.12.138.195:40285)
0811 02:52:27.671199 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0811 02:52:27.671204 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 02:52:27.671654 (+ 450us) server_negotiation.cc:407] Connection header received
0811 02:52:27.671801 (+ 147us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:52:27.671806 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:52:27.671854 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:52:27.671940 (+ 86us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 02:52:27.672229 (+ 289us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:52:27.673014 (+ 785us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:52:27.674654 (+ 1640us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:52:27.675246 (+ 592us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:52:27.675408 (+ 162us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:52:27.675458 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0811 02:52:27.675646 (+ 188us) server_negotiation.cc:299] Negotiation successful
0811 02:52:27.675719 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260811 02:52:27.675861 22371 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:52:27.670920 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.138.195:40285 (local address 127.12.138.193:33107)
0811 02:52:27.671570 (+ 650us) negotiation.cc:107] Waiting for socket to connect
0811 02:52:27.671584 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 02:52:27.671679 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:52:27.671954 (+ 275us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:52:27.671957 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:52:27.671974 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 02:52:27.672085 (+ 111us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:52:27.672093 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:52:27.673251 (+ 1158us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:52:27.673257 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:52:27.674514 (+ 1257us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:52:27.674527 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:52:27.675403 (+ 876us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:52:27.675407 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:52:27.675501 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:52:27.675524 (+ 23us) client_negotiation.cc:769] Sending connection context
0811 02:52:27.675624 (+ 100us) client_negotiation.cc:240] Negotiation successful
0811 02:52:27.675699 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":582,"mutex_wait_us":341,"thread_start_us":90,"threads_started":1}
I20260811 02:52:27.676322 22249 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a31cde49c96b46f5b5716dcda2590837" candidate_uuid: "6937bfa1e57344d78f8cc3b5df278e87" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6a04a1a9c41e4f7fb9064c78468ba498" is_pre_election: true
I20260811 02:52:27.676499 22249 raft_consensus.cc:2454] T a31cde49c96b46f5b5716dcda2590837 P 6a04a1a9c41e4f7fb9064c78468ba498 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6937bfa1e57344d78f8cc3b5df278e87 in term 0.
I20260811 02:52:27.676522 22370 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:52:27.670920 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.138.194:39121 (local address 127.12.138.193:45391)
0811 02:52:27.671340 (+ 420us) negotiation.cc:107] Waiting for socket to connect
0811 02:52:27.671358 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 02:52:27.671486 (+ 128us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:52:27.672048 (+ 562us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:52:27.672051 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:52:27.672065 (+ 14us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 02:52:27.672172 (+ 107us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:52:27.672178 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:52:27.673075 (+ 897us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:52:27.673079 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:52:27.674303 (+ 1224us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:52:27.674314 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:52:27.675156 (+ 842us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:52:27.675160 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:52:27.675289 (+ 129us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:52:27.675319 (+ 30us) client_negotiation.cc:769] Sending connection context
0811 02:52:27.676290 (+ 971us) client_negotiation.cc:240] Negotiation successful
0811 02:52:27.676368 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":295,"mutex_wait_us":8,"thread_start_us":97,"threads_started":1}
I20260811 02:52:27.676900 21860 leader_election.cc:302] T a31cde49c96b46f5b5716dcda2590837 P 6937bfa1e57344d78f8cc3b5df278e87 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6937bfa1e57344d78f8cc3b5df278e87, 6a04a1a9c41e4f7fb9064c78468ba498; no voters:
I20260811 02:52:27.677103 22093 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a31cde49c96b46f5b5716dcda2590837" candidate_uuid: "6937bfa1e57344d78f8cc3b5df278e87" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6c8a58fdbe944622b27ade12cd92a62e" is_pre_election: true
I20260811 02:52:27.677201 22346 raft_consensus.cc:2781] T a31cde49c96b46f5b5716dcda2590837 P 6937bfa1e57344d78f8cc3b5df278e87 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 02:52:27.677289 22346 raft_consensus.cc:482] T a31cde49c96b46f5b5716dcda2590837 P 6937bfa1e57344d78f8cc3b5df278e87 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 02:52:27.677340 22346 raft_consensus.cc:3037] T a31cde49c96b46f5b5716dcda2590837 P 6937bfa1e57344d78f8cc3b5df278e87 [term 0 FOLLOWER]: Advancing to term 1
I20260811 02:52:27.677369 22093 raft_consensus.cc:2454] T a31cde49c96b46f5b5716dcda2590837 P 6c8a58fdbe944622b27ade12cd92a62e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6937bfa1e57344d78f8cc3b5df278e87 in term 0.
I20260811 02:52:27.678460 22346 raft_consensus.cc:504] T a31cde49c96b46f5b5716dcda2590837 P 6937bfa1e57344d78f8cc3b5df278e87 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6937bfa1e57344d78f8cc3b5df278e87" member_type: VOTER last_known_addr { host: "127.12.138.193" port: 34941 } } peers { permanent_uuid: "6c8a58fdbe944622b27ade12cd92a62e" member_type: VOTER last_known_addr { host: "127.12.138.194" port: 39121 } } peers { permanent_uuid: "6a04a1a9c41e4f7fb9064c78468ba498" member_type: VOTER last_known_addr { host: "127.12.138.195" port: 40285 } }
I20260811 02:52:27.678900 22346 leader_election.cc:288] T a31cde49c96b46f5b5716dcda2590837 P 6937bfa1e57344d78f8cc3b5df278e87 [CANDIDATE]: Term 1 election: Requested vote from peers 6c8a58fdbe944622b27ade12cd92a62e (127.12.138.194:39121), 6a04a1a9c41e4f7fb9064c78468ba498 (127.12.138.195:40285)
I20260811 02:52:27.679344 22093 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a31cde49c96b46f5b5716dcda2590837" candidate_uuid: "6937bfa1e57344d78f8cc3b5df278e87" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6c8a58fdbe944622b27ade12cd92a62e"
I20260811 02:52:27.679463 22093 raft_consensus.cc:3037] T a31cde49c96b46f5b5716dcda2590837 P 6c8a58fdbe944622b27ade12cd92a62e [term 0 FOLLOWER]: Advancing to term 1
I20260811 02:52:27.679575 22249 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a31cde49c96b46f5b5716dcda2590837" candidate_uuid: "6937bfa1e57344d78f8cc3b5df278e87" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6a04a1a9c41e4f7fb9064c78468ba498"
I20260811 02:52:27.679723 22249 raft_consensus.cc:3037] T a31cde49c96b46f5b5716dcda2590837 P 6a04a1a9c41e4f7fb9064c78468ba498 [term 0 FOLLOWER]: Advancing to term 1
I20260811 02:52:27.680590 22249 raft_consensus.cc:2454] T a31cde49c96b46f5b5716dcda2590837 P 6a04a1a9c41e4f7fb9064c78468ba498 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6937bfa1e57344d78f8cc3b5df278e87 in term 1.
I20260811 02:52:27.680624 22093 raft_consensus.cc:2454] T a31cde49c96b46f5b5716dcda2590837 P 6c8a58fdbe944622b27ade12cd92a62e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6937bfa1e57344d78f8cc3b5df278e87 in term 1.
I20260811 02:52:27.680924 21860 leader_election.cc:302] T a31cde49c96b46f5b5716dcda2590837 P 6937bfa1e57344d78f8cc3b5df278e87 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6937bfa1e57344d78f8cc3b5df278e87, 6a04a1a9c41e4f7fb9064c78468ba498; no voters:
I20260811 02:52:27.681099 22346 raft_consensus.cc:2781] T a31cde49c96b46f5b5716dcda2590837 P 6937bfa1e57344d78f8cc3b5df278e87 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 02:52:27.681440 22346 raft_consensus.cc:686] T a31cde49c96b46f5b5716dcda2590837 P 6937bfa1e57344d78f8cc3b5df278e87 [term 1 LEADER]: Becoming Leader. State: Replica: 6937bfa1e57344d78f8cc3b5df278e87, State: Running, Role: LEADER
I20260811 02:52:27.681739 22346 consensus_queue.cc:237] T a31cde49c96b46f5b5716dcda2590837 P 6937bfa1e57344d78f8cc3b5df278e87 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6937bfa1e57344d78f8cc3b5df278e87" member_type: VOTER last_known_addr { host: "127.12.138.193" port: 34941 } } peers { permanent_uuid: "6c8a58fdbe944622b27ade12cd92a62e" member_type: VOTER last_known_addr { host: "127.12.138.194" port: 39121 } } peers { permanent_uuid: "6a04a1a9c41e4f7fb9064c78468ba498" member_type: VOTER last_known_addr { host: "127.12.138.195" port: 40285 } }
I20260811 02:52:27.684294 21761 catalog_manager.cc:5495] T a31cde49c96b46f5b5716dcda2590837 P 6937bfa1e57344d78f8cc3b5df278e87 reported cstate change: term changed from 0 to 1, leader changed from <none> to 6937bfa1e57344d78f8cc3b5df278e87 (127.12.138.193). New cstate: current_term: 1 leader_uuid: "6937bfa1e57344d78f8cc3b5df278e87" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6937bfa1e57344d78f8cc3b5df278e87" member_type: VOTER last_known_addr { host: "127.12.138.193" port: 34941 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6c8a58fdbe944622b27ade12cd92a62e" member_type: VOTER last_known_addr { host: "127.12.138.194" port: 39121 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6a04a1a9c41e4f7fb9064c78468ba498" member_type: VOTER last_known_addr { host: "127.12.138.195" port: 40285 } health_report { overall_health: UNKNOWN } } }
Aug 11 02:52:27 dist-test-slave-9gcw krb5kdc[21711](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416746, etypes {rep=17 tkt=17 ses=17}, oryx/127.12.138.254@KRBTEST.COM for oryx/127.12.138.254@KRBTEST.COM
I20260811 02:52:27.719568 22379 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:52:27.711138 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.138.254:43149 (local address 127.0.0.1:36692)
0811 02:52:27.711458 (+ 320us) negotiation.cc:107] Waiting for socket to connect
0811 02:52:27.711477 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0811 02:52:27.711589 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:52:27.711857 (+ 268us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:52:27.711860 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:52:27.712261 (+ 401us) client_negotiation.cc:189] Negotiated authn=SASL
0811 02:52:27.712388 (+ 127us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:52:27.712395 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:52:27.713192 (+ 797us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:52:27.713194 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:52:27.713756 (+ 562us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:52:27.713764 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:52:27.714177 (+ 413us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:52:27.714179 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:52:27.714241 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:52:27.714809 (+ 568us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 02:52:27.714822 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0811 02:52:27.716345 (+ 1523us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 02:52:27.718156 (+ 1811us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:52:27.718165 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:52:27.718170 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:52:27.718405 (+ 235us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:52:27.718656 (+ 251us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:52:27.718658 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:52:27.718659 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:52:27.718740 (+ 81us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:52:27.719088 (+ 348us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 02:52:27.719094 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 02:52:27.719202 (+ 108us) client_negotiation.cc:769] Sending connection context
0811 02:52:27.719294 (+ 92us) client_negotiation.cc:240] Negotiation successful
0811 02:52:27.719423 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":212,"thread_start_us":128,"threads_started":1}
I20260811 02:52:27.719659 22380 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:52:27.711239 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36692 (local address 127.12.138.254:43149)
0811 02:52:27.711526 (+ 287us) server_negotiation.cc:206] Beginning negotiation
0811 02:52:27.711533 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 02:52:27.711551 (+ 18us) server_negotiation.cc:407] Connection header received
0811 02:52:27.711681 (+ 130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:52:27.711684 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:52:27.711725 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:52:27.711821 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:52:27.712500 (+ 679us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:52:27.713065 (+ 565us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:52:27.713857 (+ 792us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:52:27.714075 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:52:27.714153 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:52:27.716473 (+ 2320us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:52:27.716476 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:52:27.716478 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:52:27.716499 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:52:27.718063 (+ 1564us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:52:27.718492 (+ 429us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:52:27.718495 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:52:27.718496 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:52:27.718574 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:52:27.718823 (+ 249us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:52:27.718825 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:52:27.718826 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:52:27.719000 (+ 174us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:52:27.719074 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0811 02:52:27.719354 (+ 280us) server_negotiation.cc:299] Negotiation successful
0811 02:52:27.719512 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":20,"server-negotiator.queue_time_us":214,"thread_start_us":139,"threads_started":1}
I20260811 02:52:27.723835 21761 catalog_manager.cc:1908] Servicing CreateTable request from {username='oryx', principal='oryx/127.12.138.254@KRBTEST.COM'} at 127.0.0.1:36692:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260811 02:52:27.724587 21761 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 02:52:27.729944 21916 tablet_service.cc:1467] Processing CreateTablet for tablet ed250b7a112a47959cd6a29b3089a345 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=82f78b952812401f9f07a91d74e624df]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260811 02:52:27.729997 22073 tablet_service.cc:1467] Processing CreateTablet for tablet ed250b7a112a47959cd6a29b3089a345 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=82f78b952812401f9f07a91d74e624df]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260811 02:52:27.730048 22225 tablet_service.cc:1467] Processing CreateTablet for tablet ed250b7a112a47959cd6a29b3089a345 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=82f78b952812401f9f07a91d74e624df]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260811 02:52:27.730381 22073 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ed250b7a112a47959cd6a29b3089a345. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 02:52:27.730410 21916 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ed250b7a112a47959cd6a29b3089a345. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 02:52:27.730499 22225 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ed250b7a112a47959cd6a29b3089a345. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 02:52:27.733042 22337 tablet_bootstrap.cc:492] T ed250b7a112a47959cd6a29b3089a345 P 6a04a1a9c41e4f7fb9064c78468ba498: Bootstrap starting.
I20260811 02:52:27.733266 22340 tablet_bootstrap.cc:492] T ed250b7a112a47959cd6a29b3089a345 P 6c8a58fdbe944622b27ade12cd92a62e: Bootstrap starting.
I20260811 02:52:27.733275 22338 tablet_bootstrap.cc:492] T ed250b7a112a47959cd6a29b3089a345 P 6937bfa1e57344d78f8cc3b5df278e87: Bootstrap starting.
I20260811 02:52:27.734050 22337 tablet_bootstrap.cc:654] T ed250b7a112a47959cd6a29b3089a345 P 6a04a1a9c41e4f7fb9064c78468ba498: Neither blocks nor log segments found. Creating new log.
I20260811 02:52:27.734050 22340 tablet_bootstrap.cc:654] T ed250b7a112a47959cd6a29b3089a345 P 6c8a58fdbe944622b27ade12cd92a62e: Neither blocks nor log segments found. Creating new log.
I20260811 02:52:27.734223 22338 tablet_bootstrap.cc:654] T ed250b7a112a47959cd6a29b3089a345 P 6937bfa1e57344d78f8cc3b5df278e87: Neither blocks nor log segments found. Creating new log.
I20260811 02:52:27.735410 22340 tablet_bootstrap.cc:492] T ed250b7a112a47959cd6a29b3089a345 P 6c8a58fdbe944622b27ade12cd92a62e: No bootstrap required, opened a new log
I20260811 02:52:27.735435 22338 tablet_bootstrap.cc:492] T ed250b7a112a47959cd6a29b3089a345 P 6937bfa1e57344d78f8cc3b5df278e87: No bootstrap required, opened a new log
I20260811 02:52:27.735505 22340 ts_tablet_manager.cc:1397] T ed250b7a112a47959cd6a29b3089a345 P 6c8a58fdbe944622b27ade12cd92a62e: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260811 02:52:27.735533 22338 ts_tablet_manager.cc:1397] T ed250b7a112a47959cd6a29b3089a345 P 6937bfa1e57344d78f8cc3b5df278e87: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260811 02:52:27.735661 22337 tablet_bootstrap.cc:492] T ed250b7a112a47959cd6a29b3089a345 P 6a04a1a9c41e4f7fb9064c78468ba498: No bootstrap required, opened a new log
I20260811 02:52:27.735733 22337 ts_tablet_manager.cc:1397] T ed250b7a112a47959cd6a29b3089a345 P 6a04a1a9c41e4f7fb9064c78468ba498: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260811 02:52:27.735966 22338 raft_consensus.cc:348] T ed250b7a112a47959cd6a29b3089a345 P 6937bfa1e57344d78f8cc3b5df278e87 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6937bfa1e57344d78f8cc3b5df278e87" member_type: VOTER last_known_addr { host: "127.12.138.193" port: 34941 } } peers { permanent_uuid: "6c8a58fdbe944622b27ade12cd92a62e" member_type: VOTER last_known_addr { host: "127.12.138.194" port: 39121 } } peers { permanent_uuid: "6a04a1a9c41e4f7fb9064c78468ba498" member_type: VOTER last_known_addr { host: "127.12.138.195" port: 40285 } }
I20260811 02:52:27.735970 22340 raft_consensus.cc:348] T ed250b7a112a47959cd6a29b3089a345 P 6c8a58fdbe944622b27ade12cd92a62e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6937bfa1e57344d78f8cc3b5df278e87" member_type: VOTER last_known_addr { host: "127.12.138.193" port: 34941 } } peers { permanent_uuid: "6c8a58fdbe944622b27ade12cd92a62e" member_type: VOTER last_known_addr { host: "127.12.138.194" port: 39121 } } peers { permanent_uuid: "6a04a1a9c41e4f7fb9064c78468ba498" member_type: VOTER last_known_addr { host: "127.12.138.195" port: 40285 } }
I20260811 02:52:27.736079 22340 raft_consensus.cc:374] T ed250b7a112a47959cd6a29b3089a345 P 6c8a58fdbe944622b27ade12cd92a62e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 02:52:27.736096 22338 raft_consensus.cc:374] T ed250b7a112a47959cd6a29b3089a345 P 6937bfa1e57344d78f8cc3b5df278e87 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 02:52:27.736112 22340 raft_consensus.cc:729] T ed250b7a112a47959cd6a29b3089a345 P 6c8a58fdbe944622b27ade12cd92a62e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6c8a58fdbe944622b27ade12cd92a62e, State: Initialized, Role: FOLLOWER
I20260811 02:52:27.736140 22338 raft_consensus.cc:729] T ed250b7a112a47959cd6a29b3089a345 P 6937bfa1e57344d78f8cc3b5df278e87 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6937bfa1e57344d78f8cc3b5df278e87, State: Initialized, Role: FOLLOWER
I20260811 02:52:27.736150 22337 raft_consensus.cc:348] T ed250b7a112a47959cd6a29b3089a345 P 6a04a1a9c41e4f7fb9064c78468ba498 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6937bfa1e57344d78f8cc3b5df278e87" member_type: VOTER last_known_addr { host: "127.12.138.193" port: 34941 } } peers { permanent_uuid: "6c8a58fdbe944622b27ade12cd92a62e" member_type: VOTER last_known_addr { host: "127.12.138.194" port: 39121 } } peers { permanent_uuid: "6a04a1a9c41e4f7fb9064c78468ba498" member_type: VOTER last_known_addr { host: "127.12.138.195" port: 40285 } }
I20260811 02:52:27.736263 22337 raft_consensus.cc:374] T ed250b7a112a47959cd6a29b3089a345 P 6a04a1a9c41e4f7fb9064c78468ba498 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 02:52:27.736243 22340 consensus_queue.cc:260] T ed250b7a112a47959cd6a29b3089a345 P 6c8a58fdbe944622b27ade12cd92a62e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6937bfa1e57344d78f8cc3b5df278e87" member_type: VOTER last_known_addr { host: "127.12.138.193" port: 34941 } } peers { permanent_uuid: "6c8a58fdbe944622b27ade12cd92a62e" member_type: VOTER last_known_addr { host: "127.12.138.194" port: 39121 } } peers { permanent_uuid: "6a04a1a9c41e4f7fb9064c78468ba498" member_type: VOTER last_known_addr { host: "127.12.138.195" port: 40285 } }
I20260811 02:52:27.736302 22337 raft_consensus.cc:729] T ed250b7a112a47959cd6a29b3089a345 P 6a04a1a9c41e4f7fb9064c78468ba498 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6a04a1a9c41e4f7fb9064c78468ba498, State: Initialized, Role: FOLLOWER
I20260811 02:52:27.736300 22338 consensus_queue.cc:260] T ed250b7a112a47959cd6a29b3089a345 P 6937bfa1e57344d78f8cc3b5df278e87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6937bfa1e57344d78f8cc3b5df278e87" member_type: VOTER last_known_addr { host: "127.12.138.193" port: 34941 } } peers { permanent_uuid: "6c8a58fdbe944622b27ade12cd92a62e" member_type: VOTER last_known_addr { host: "127.12.138.194" port: 39121 } } peers { permanent_uuid: "6a04a1a9c41e4f7fb9064c78468ba498" member_type: VOTER last_known_addr { host: "127.12.138.195" port: 40285 } }
I20260811 02:52:27.736416 22337 consensus_queue.cc:260] T ed250b7a112a47959cd6a29b3089a345 P 6a04a1a9c41e4f7fb9064c78468ba498 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6937bfa1e57344d78f8cc3b5df278e87" member_type: VOTER last_known_addr { host: "127.12.138.193" port: 34941 } } peers { permanent_uuid: "6c8a58fdbe944622b27ade12cd92a62e" member_type: VOTER last_known_addr { host: "127.12.138.194" port: 39121 } } peers { permanent_uuid: "6a04a1a9c41e4f7fb9064c78468ba498" member_type: VOTER last_known_addr { host: "127.12.138.195" port: 40285 } }
I20260811 02:52:27.736533 22340 ts_tablet_manager.cc:1428] T ed250b7a112a47959cd6a29b3089a345 P 6c8a58fdbe944622b27ade12cd92a62e: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260811 02:52:27.736644 22338 ts_tablet_manager.cc:1428] T ed250b7a112a47959cd6a29b3089a345 P 6937bfa1e57344d78f8cc3b5df278e87: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260811 02:52:27.736680 22337 ts_tablet_manager.cc:1428] T ed250b7a112a47959cd6a29b3089a345 P 6a04a1a9c41e4f7fb9064c78468ba498: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260811 02:52:27.736739 22345 tablet_replica.cc:439] T ed250b7a112a47959cd6a29b3089a345 P 6c8a58fdbe944622b27ade12cd92a62e: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6937bfa1e57344d78f8cc3b5df278e87" member_type: VOTER last_known_addr { host: "127.12.138.193" port: 34941 } } peers { permanent_uuid: "6c8a58fdbe944622b27ade12cd92a62e" member_type: VOTER last_known_addr { host: "127.12.138.194" port: 39121 } } peers { permanent_uuid: "6a04a1a9c41e4f7fb9064c78468ba498" member_type: VOTER last_known_addr { host: "127.12.138.195" port: 40285 } } }
I20260811 02:52:27.736871 22345 tablet_replica.cc:442] T ed250b7a112a47959cd6a29b3089a345 P 6c8a58fdbe944622b27ade12cd92a62e: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 02:52:27.736848 22344 tablet_replica.cc:439] T ed250b7a112a47959cd6a29b3089a345 P 6a04a1a9c41e4f7fb9064c78468ba498: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6937bfa1e57344d78f8cc3b5df278e87" member_type: VOTER last_known_addr { host: "127.12.138.193" port: 34941 } } peers { permanent_uuid: "6c8a58fdbe944622b27ade12cd92a62e" member_type: VOTER last_known_addr { host: "127.12.138.194" port: 39121 } } peers { permanent_uuid: "6a04a1a9c41e4f7fb9064c78468ba498" member_type: VOTER last_known_addr { host: "127.12.138.195" port: 40285 } } }
I20260811 02:52:27.736967 22344 tablet_replica.cc:442] T ed250b7a112a47959cd6a29b3089a345 P 6a04a1a9c41e4f7fb9064c78468ba498: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 02:52:27.737049 22346 tablet_replica.cc:439] T ed250b7a112a47959cd6a29b3089a345 P 6937bfa1e57344d78f8cc3b5df278e87: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6937bfa1e57344d78f8cc3b5df278e87" member_type: VOTER last_known_addr { host: "127.12.138.193" port: 34941 } } peers { permanent_uuid: "6c8a58fdbe944622b27ade12cd92a62e" member_type: VOTER last_known_addr { host: "127.12.138.194" port: 39121 } } peers { permanent_uuid: "6a04a1a9c41e4f7fb9064c78468ba498" member_type: VOTER last_known_addr { host: "127.12.138.195" port: 40285 } } }
I20260811 02:52:27.737192 22346 tablet_replica.cc:442] T ed250b7a112a47959cd6a29b3089a345 P 6937bfa1e57344d78f8cc3b5df278e87: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 02:52:27.913033 22346 raft_consensus.cc:482] T ed250b7a112a47959cd6a29b3089a345 P 6937bfa1e57344d78f8cc3b5df278e87 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 02:52:27.913260 22346 raft_consensus.cc:504] T ed250b7a112a47959cd6a29b3089a345 P 6937bfa1e57344d78f8cc3b5df278e87 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6937bfa1e57344d78f8cc3b5df278e87" member_type: VOTER last_known_addr { host: "127.12.138.193" port: 34941 } } peers { permanent_uuid: "6c8a58fdbe944622b27ade12cd92a62e" member_type: VOTER last_known_addr { host: "127.12.138.194" port: 39121 } } peers { permanent_uuid: "6a04a1a9c41e4f7fb9064c78468ba498" member_type: VOTER last_known_addr { host: "127.12.138.195" port: 40285 } }
I20260811 02:52:27.913751 22346 leader_election.cc:288] T ed250b7a112a47959cd6a29b3089a345 P 6937bfa1e57344d78f8cc3b5df278e87 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6c8a58fdbe944622b27ade12cd92a62e (127.12.138.194:39121), 6a04a1a9c41e4f7fb9064c78468ba498 (127.12.138.195:40285)
I20260811 02:52:27.914346 22249 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ed250b7a112a47959cd6a29b3089a345" candidate_uuid: "6937bfa1e57344d78f8cc3b5df278e87" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6a04a1a9c41e4f7fb9064c78468ba498" is_pre_election: true
I20260811 02:52:27.914350 22093 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ed250b7a112a47959cd6a29b3089a345" candidate_uuid: "6937bfa1e57344d78f8cc3b5df278e87" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6c8a58fdbe944622b27ade12cd92a62e" is_pre_election: true
I20260811 02:52:27.914518 22249 raft_consensus.cc:2454] T ed250b7a112a47959cd6a29b3089a345 P 6a04a1a9c41e4f7fb9064c78468ba498 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6937bfa1e57344d78f8cc3b5df278e87 in term 0.
I20260811 02:52:27.914533 22093 raft_consensus.cc:2454] T ed250b7a112a47959cd6a29b3089a345 P 6c8a58fdbe944622b27ade12cd92a62e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6937bfa1e57344d78f8cc3b5df278e87 in term 0.
I20260811 02:52:27.914911 21860 leader_election.cc:302] T ed250b7a112a47959cd6a29b3089a345 P 6937bfa1e57344d78f8cc3b5df278e87 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6937bfa1e57344d78f8cc3b5df278e87, 6a04a1a9c41e4f7fb9064c78468ba498; no voters:
I20260811 02:52:27.915145 22346 raft_consensus.cc:2781] T ed250b7a112a47959cd6a29b3089a345 P 6937bfa1e57344d78f8cc3b5df278e87 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 02:52:27.915213 22346 raft_consensus.cc:482] T ed250b7a112a47959cd6a29b3089a345 P 6937bfa1e57344d78f8cc3b5df278e87 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 02:52:27.915266 22346 raft_consensus.cc:3037] T ed250b7a112a47959cd6a29b3089a345 P 6937bfa1e57344d78f8cc3b5df278e87 [term 0 FOLLOWER]: Advancing to term 1
I20260811 02:52:27.916289 22346 raft_consensus.cc:504] T ed250b7a112a47959cd6a29b3089a345 P 6937bfa1e57344d78f8cc3b5df278e87 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6937bfa1e57344d78f8cc3b5df278e87" member_type: VOTER last_known_addr { host: "127.12.138.193" port: 34941 } } peers { permanent_uuid: "6c8a58fdbe944622b27ade12cd92a62e" member_type: VOTER last_known_addr { host: "127.12.138.194" port: 39121 } } peers { permanent_uuid: "6a04a1a9c41e4f7fb9064c78468ba498" member_type: VOTER last_known_addr { host: "127.12.138.195" port: 40285 } }
I20260811 02:52:27.916656 22346 leader_election.cc:288] T ed250b7a112a47959cd6a29b3089a345 P 6937bfa1e57344d78f8cc3b5df278e87 [CANDIDATE]: Term 1 election: Requested vote from peers 6c8a58fdbe944622b27ade12cd92a62e (127.12.138.194:39121), 6a04a1a9c41e4f7fb9064c78468ba498 (127.12.138.195:40285)
I20260811 02:52:27.917016 22093 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ed250b7a112a47959cd6a29b3089a345" candidate_uuid: "6937bfa1e57344d78f8cc3b5df278e87" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6c8a58fdbe944622b27ade12cd92a62e"
I20260811 02:52:27.917068 22249 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ed250b7a112a47959cd6a29b3089a345" candidate_uuid: "6937bfa1e57344d78f8cc3b5df278e87" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6a04a1a9c41e4f7fb9064c78468ba498"
I20260811 02:52:27.917114 22093 raft_consensus.cc:3037] T ed250b7a112a47959cd6a29b3089a345 P 6c8a58fdbe944622b27ade12cd92a62e [term 0 FOLLOWER]: Advancing to term 1
I20260811 02:52:27.917248 22249 raft_consensus.cc:3037] T ed250b7a112a47959cd6a29b3089a345 P 6a04a1a9c41e4f7fb9064c78468ba498 [term 0 FOLLOWER]: Advancing to term 1
I20260811 02:52:27.918181 22093 raft_consensus.cc:2454] T ed250b7a112a47959cd6a29b3089a345 P 6c8a58fdbe944622b27ade12cd92a62e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6937bfa1e57344d78f8cc3b5df278e87 in term 1.
I20260811 02:52:27.918184 22249 raft_consensus.cc:2454] T ed250b7a112a47959cd6a29b3089a345 P 6a04a1a9c41e4f7fb9064c78468ba498 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6937bfa1e57344d78f8cc3b5df278e87 in term 1.
I20260811 02:52:27.918546 21860 leader_election.cc:302] T ed250b7a112a47959cd6a29b3089a345 P 6937bfa1e57344d78f8cc3b5df278e87 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 6937bfa1e57344d78f8cc3b5df278e87, 6a04a1a9c41e4f7fb9064c78468ba498, 6c8a58fdbe944622b27ade12cd92a62e; no voters:
I20260811 02:52:27.918782 22346 raft_consensus.cc:2781] T ed250b7a112a47959cd6a29b3089a345 P 6937bfa1e57344d78f8cc3b5df278e87 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 02:52:27.918885 22346 raft_consensus.cc:686] T ed250b7a112a47959cd6a29b3089a345 P 6937bfa1e57344d78f8cc3b5df278e87 [term 1 LEADER]: Becoming Leader. State: Replica: 6937bfa1e57344d78f8cc3b5df278e87, State: Running, Role: LEADER
I20260811 02:52:27.919019 22346 consensus_queue.cc:237] T ed250b7a112a47959cd6a29b3089a345 P 6937bfa1e57344d78f8cc3b5df278e87 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6937bfa1e57344d78f8cc3b5df278e87" member_type: VOTER last_known_addr { host: "127.12.138.193" port: 34941 } } peers { permanent_uuid: "6c8a58fdbe944622b27ade12cd92a62e" member_type: VOTER last_known_addr { host: "127.12.138.194" port: 39121 } } peers { permanent_uuid: "6a04a1a9c41e4f7fb9064c78468ba498" member_type: VOTER last_known_addr { host: "127.12.138.195" port: 40285 } }
I20260811 02:52:27.919762 22372 tablet_replica.cc:439] T ed250b7a112a47959cd6a29b3089a345 P 6937bfa1e57344d78f8cc3b5df278e87: TxnStatusTablet state changed. Reason: New leader 6937bfa1e57344d78f8cc3b5df278e87. Latest consensus state: current_term: 1 leader_uuid: "6937bfa1e57344d78f8cc3b5df278e87" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6937bfa1e57344d78f8cc3b5df278e87" member_type: VOTER last_known_addr { host: "127.12.138.193" port: 34941 } } peers { permanent_uuid: "6c8a58fdbe944622b27ade12cd92a62e" member_type: VOTER last_known_addr { host: "127.12.138.194" port: 39121 } } peers { permanent_uuid: "6a04a1a9c41e4f7fb9064c78468ba498" member_type: VOTER last_known_addr { host: "127.12.138.195" port: 40285 } } }
I20260811 02:52:27.919886 22372 tablet_replica.cc:442] T ed250b7a112a47959cd6a29b3089a345 P 6937bfa1e57344d78f8cc3b5df278e87: This TxnStatusTablet replica's current role is: LEADER
I20260811 02:52:27.920655 22386 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260811 02:52:27.920830 22386 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260811 02:52:27.920997 21760 catalog_manager.cc:5495] T ed250b7a112a47959cd6a29b3089a345 P 6937bfa1e57344d78f8cc3b5df278e87 reported cstate change: term changed from 0 to 1, leader changed from <none> to 6937bfa1e57344d78f8cc3b5df278e87 (127.12.138.193). New cstate: current_term: 1 leader_uuid: "6937bfa1e57344d78f8cc3b5df278e87" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6937bfa1e57344d78f8cc3b5df278e87" member_type: VOTER last_known_addr { host: "127.12.138.193" port: 34941 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6c8a58fdbe944622b27ade12cd92a62e" member_type: VOTER last_known_addr { host: "127.12.138.194" port: 39121 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6a04a1a9c41e4f7fb9064c78468ba498" member_type: VOTER last_known_addr { host: "127.12.138.195" port: 40285 } health_report { overall_health: UNKNOWN } } }
I20260811 02:52:27.978511 22389 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:52:27.974464 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35374 (local address 127.12.138.193:34941)
0811 02:52:27.974885 (+ 421us) server_negotiation.cc:206] Beginning negotiation
0811 02:52:27.974889 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 02:52:27.974901 (+ 12us) server_negotiation.cc:407] Connection header received
0811 02:52:27.974978 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:52:27.974980 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:52:27.975036 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:52:27.975162 (+ 126us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 02:52:27.975434 (+ 272us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:52:27.976040 (+ 606us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:52:27.976905 (+ 865us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:52:27.977206 (+ 301us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:52:27.977314 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:52:27.977616 (+ 302us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 02:52:27.978061 (+ 445us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 02:52:27.978190 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0811 02:52:27.978275 (+ 85us) server_negotiation.cc:299] Negotiation successful
0811 02:52:27.978333 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":341,"thread_start_us":86,"threads_started":1}
I20260811 02:52:27.978499 22379 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:52:27.974326 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.138.193:34941 (local address 127.0.0.1:35374)
0811 02:52:27.974515 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0811 02:52:27.974531 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 02:52:27.974651 (+ 120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:52:27.975177 (+ 526us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:52:27.975180 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:52:27.975188 (+ 8us) client_negotiation.cc:189] Negotiated authn=TOKEN
0811 02:52:27.975310 (+ 122us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:52:27.975316 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:52:27.976160 (+ 844us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:52:27.976163 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:52:27.976775 (+ 612us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:52:27.976783 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:52:27.977322 (+ 539us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:52:27.977325 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:52:27.977415 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:52:27.977461 (+ 46us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0811 02:52:27.978165 (+ 704us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0811 02:52:27.978182 (+ 17us) client_negotiation.cc:769] Sending connection context
0811 02:52:27.978256 (+ 74us) client_negotiation.cc:240] Negotiation successful
0811 02:52:27.978352 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":66}
I20260811 02:52:27.981679 22249 raft_consensus.cc:1261] T ed250b7a112a47959cd6a29b3089a345 P 6a04a1a9c41e4f7fb9064c78468ba498 [term 1 FOLLOWER]: Refusing update from remote peer 6937bfa1e57344d78f8cc3b5df278e87: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 02:52:27.981875 22093 raft_consensus.cc:1261] T ed250b7a112a47959cd6a29b3089a345 P 6c8a58fdbe944622b27ade12cd92a62e [term 1 FOLLOWER]: Refusing update from remote peer 6937bfa1e57344d78f8cc3b5df278e87: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 02:52:27.982239 22372 consensus_queue.cc:1035] T ed250b7a112a47959cd6a29b3089a345 P 6937bfa1e57344d78f8cc3b5df278e87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a04a1a9c41e4f7fb9064c78468ba498" member_type: VOTER last_known_addr { host: "127.12.138.195" port: 40285 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 02:52:27.982453 22346 consensus_queue.cc:1035] T ed250b7a112a47959cd6a29b3089a345 P 6937bfa1e57344d78f8cc3b5df278e87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c8a58fdbe944622b27ade12cd92a62e" member_type: VOTER last_known_addr { host: "127.12.138.194" port: 39121 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 02:52:27.984149 22345 tablet_replica.cc:439] T ed250b7a112a47959cd6a29b3089a345 P 6c8a58fdbe944622b27ade12cd92a62e: TxnStatusTablet state changed. Reason: New leader 6937bfa1e57344d78f8cc3b5df278e87. Latest consensus state: current_term: 1 leader_uuid: "6937bfa1e57344d78f8cc3b5df278e87" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6937bfa1e57344d78f8cc3b5df278e87" member_type: VOTER last_known_addr { host: "127.12.138.193" port: 34941 } } peers { permanent_uuid: "6c8a58fdbe944622b27ade12cd92a62e" member_type: VOTER last_known_addr { host: "127.12.138.194" port: 39121 } } peers { permanent_uuid: "6a04a1a9c41e4f7fb9064c78468ba498" member_type: VOTER last_known_addr { host: "127.12.138.195" port: 40285 } } }
I20260811 02:52:27.984266 22345 tablet_replica.cc:442] T ed250b7a112a47959cd6a29b3089a345 P 6c8a58fdbe944622b27ade12cd92a62e: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 02:52:27.985929 22344 tablet_replica.cc:439] T ed250b7a112a47959cd6a29b3089a345 P 6a04a1a9c41e4f7fb9064c78468ba498: TxnStatusTablet state changed. Reason: New leader 6937bfa1e57344d78f8cc3b5df278e87. Latest consensus state: current_term: 1 leader_uuid: "6937bfa1e57344d78f8cc3b5df278e87" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6937bfa1e57344d78f8cc3b5df278e87" member_type: VOTER last_known_addr { host: "127.12.138.193" port: 34941 } } peers { permanent_uuid: "6c8a58fdbe944622b27ade12cd92a62e" member_type: VOTER last_known_addr { host: "127.12.138.194" port: 39121 } } peers { permanent_uuid: "6a04a1a9c41e4f7fb9064c78468ba498" member_type: VOTER last_known_addr { host: "127.12.138.195" port: 40285 } } }
I20260811 02:52:27.986053 22344 tablet_replica.cc:442] T ed250b7a112a47959cd6a29b3089a345 P 6a04a1a9c41e4f7fb9064c78468ba498: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 02:52:27.987093 22372 tablet_replica.cc:439] T ed250b7a112a47959cd6a29b3089a345 P 6937bfa1e57344d78f8cc3b5df278e87: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "6937bfa1e57344d78f8cc3b5df278e87" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6937bfa1e57344d78f8cc3b5df278e87" member_type: VOTER last_known_addr { host: "127.12.138.193" port: 34941 } } peers { permanent_uuid: "6c8a58fdbe944622b27ade12cd92a62e" member_type: VOTER last_known_addr { host: "127.12.138.194" port: 39121 } } peers { permanent_uuid: "6a04a1a9c41e4f7fb9064c78468ba498" member_type: VOTER last_known_addr { host: "127.12.138.195" port: 40285 } } }
I20260811 02:52:27.987202 22372 tablet_replica.cc:442] T ed250b7a112a47959cd6a29b3089a345 P 6937bfa1e57344d78f8cc3b5df278e87: This TxnStatusTablet replica's current role is: LEADER
I20260811 02:52:27.987677 22344 tablet_replica.cc:439] T ed250b7a112a47959cd6a29b3089a345 P 6a04a1a9c41e4f7fb9064c78468ba498: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "6937bfa1e57344d78f8cc3b5df278e87" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6937bfa1e57344d78f8cc3b5df278e87" member_type: VOTER last_known_addr { host: "127.12.138.193" port: 34941 } } peers { permanent_uuid: "6c8a58fdbe944622b27ade12cd92a62e" member_type: VOTER last_known_addr { host: "127.12.138.194" port: 39121 } } peers { permanent_uuid: "6a04a1a9c41e4f7fb9064c78468ba498" member_type: VOTER last_known_addr { host: "127.12.138.195" port: 40285 } } }
I20260811 02:52:27.987787 22344 tablet_replica.cc:442] T ed250b7a112a47959cd6a29b3089a345 P 6a04a1a9c41e4f7fb9064c78468ba498: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 02:52:27.990670 22346 tablet_replica.cc:439] T ed250b7a112a47959cd6a29b3089a345 P 6937bfa1e57344d78f8cc3b5df278e87: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "6937bfa1e57344d78f8cc3b5df278e87" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6937bfa1e57344d78f8cc3b5df278e87" member_type: VOTER last_known_addr { host: "127.12.138.193" port: 34941 } } peers { permanent_uuid: "6c8a58fdbe944622b27ade12cd92a62e" member_type: VOTER last_known_addr { host: "127.12.138.194" port: 39121 } } peers { permanent_uuid: "6a04a1a9c41e4f7fb9064c78468ba498" member_type: VOTER last_known_addr { host: "127.12.138.195" port: 40285 } } }
I20260811 02:52:27.990798 22346 tablet_replica.cc:442] T ed250b7a112a47959cd6a29b3089a345 P 6937bfa1e57344d78f8cc3b5df278e87: This TxnStatusTablet replica's current role is: LEADER
I20260811 02:52:27.991324 22345 tablet_replica.cc:439] T ed250b7a112a47959cd6a29b3089a345 P 6c8a58fdbe944622b27ade12cd92a62e: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "6937bfa1e57344d78f8cc3b5df278e87" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6937bfa1e57344d78f8cc3b5df278e87" member_type: VOTER last_known_addr { host: "127.12.138.193" port: 34941 } } peers { permanent_uuid: "6c8a58fdbe944622b27ade12cd92a62e" member_type: VOTER last_known_addr { host: "127.12.138.194" port: 39121 } } peers { permanent_uuid: "6a04a1a9c41e4f7fb9064c78468ba498" member_type: VOTER last_known_addr { host: "127.12.138.195" port: 40285 } } }
I20260811 02:52:27.991410 22354 mvcc.cc:205] Tried to move back new op lower bound from 7317162999727632384 to 7317162999477649408. Current Snapshot: MvccSnapshot[applied={T|T < 7317162999727632384}]
I20260811 02:52:27.991456 22345 tablet_replica.cc:442] T ed250b7a112a47959cd6a29b3089a345 P 6c8a58fdbe944622b27ade12cd92a62e: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 02:52:27.996924 22389 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:52:27.993532 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35390 (local address 127.12.138.193:34941)
0811 02:52:27.993692 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0811 02:52:27.993696 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 02:52:27.993709 (+ 13us) server_negotiation.cc:407] Connection header received
0811 02:52:27.993759 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:52:27.993761 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:52:27.993799 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:52:27.993888 (+ 89us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 02:52:27.994136 (+ 248us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:52:27.994791 (+ 655us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:52:27.995759 (+ 968us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:52:27.996043 (+ 284us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:52:27.996189 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:52:27.996468 (+ 279us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 02:52:27.996547 (+ 79us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 02:52:27.996641 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0811 02:52:27.996735 (+ 94us) server_negotiation.cc:299] Negotiation successful
0811 02:52:27.996787 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260811 02:52:28.008230 22394 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:52:28.004543 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.138.193:34941 (local address 127.12.138.193:57289)
0811 02:52:28.004879 (+ 336us) negotiation.cc:107] Waiting for socket to connect
0811 02:52:28.004895 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 02:52:28.005006 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:52:28.005290 (+ 284us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:52:28.005293 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:52:28.005301 (+ 8us) client_negotiation.cc:189] Negotiated authn=TOKEN
0811 02:52:28.005414 (+ 113us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:52:28.005420 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:52:28.006124 (+ 704us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:52:28.006128 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:52:28.007067 (+ 939us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:52:28.007079 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:52:28.007494 (+ 415us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:52:28.007498 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:52:28.007585 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:52:28.007637 (+ 52us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0811 02:52:28.007928 (+ 291us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0811 02:52:28.007936 (+ 8us) client_negotiation.cc:769] Sending connection context
0811 02:52:28.008004 (+ 68us) client_negotiation.cc:240] Negotiation successful
0811 02:52:28.008089 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":233,"mutex_wait_us":3,"thread_start_us":123,"threads_started":1}
I20260811 02:52:28.008230 22389 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:52:28.004605 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.138.193:57289 (local address 127.12.138.193:34941)
0811 02:52:28.004738 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0811 02:52:28.004743 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 02:52:28.004974 (+ 231us) server_negotiation.cc:407] Connection header received
0811 02:52:28.005101 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:52:28.005103 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:52:28.005189 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:52:28.005262 (+ 73us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 02:52:28.005518 (+ 256us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:52:28.006019 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:52:28.007184 (+ 1165us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:52:28.007392 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:52:28.007467 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:52:28.007761 (+ 294us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 02:52:28.007837 (+ 76us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 02:52:28.007924 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0811 02:52:28.008011 (+ 87us) server_negotiation.cc:299] Negotiation successful
0811 02:52:28.008062 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260811 02:52:28.015297 22093 raft_consensus.cc:1261] T a31cde49c96b46f5b5716dcda2590837 P 6c8a58fdbe944622b27ade12cd92a62e [term 1 FOLLOWER]: Refusing update from remote peer 6937bfa1e57344d78f8cc3b5df278e87: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 02:52:28.015293 22249 raft_consensus.cc:1261] T a31cde49c96b46f5b5716dcda2590837 P 6a04a1a9c41e4f7fb9064c78468ba498 [term 1 FOLLOWER]: Refusing update from remote peer 6937bfa1e57344d78f8cc3b5df278e87: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 02:52:28.015710 22395 consensus_queue.cc:1035] T a31cde49c96b46f5b5716dcda2590837 P 6937bfa1e57344d78f8cc3b5df278e87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a04a1a9c41e4f7fb9064c78468ba498" member_type: VOTER last_known_addr { host: "127.12.138.195" port: 40285 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 02:52:28.015898 22346 consensus_queue.cc:1035] T a31cde49c96b46f5b5716dcda2590837 P 6937bfa1e57344d78f8cc3b5df278e87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c8a58fdbe944622b27ade12cd92a62e" member_type: VOTER last_known_addr { host: "127.12.138.194" port: 39121 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 02:52:28.067252 21760 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:36680:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 02:52:28.067441 21760 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:36680:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 02:52:28.069772 21760 catalog_manager.cc:5782] T 00000000000000000000000000000000 P acf0fa3758a14749a4a0ec3e106b3dd5: Sending DeleteTablet for 3 replicas of tablet a31cde49c96b46f5b5716dcda2590837
I20260811 02:52:28.070451 22073 tablet_service.cc:1514] Processing DeleteTablet for tablet a31cde49c96b46f5b5716dcda2590837 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 02:52:28 UTC) from {username='oryx', principal='oryx/127.12.138.254@KRBTEST.COM'} at 127.0.0.1:40316
I20260811 02:52:28.070478 22225 tablet_service.cc:1514] Processing DeleteTablet for tablet a31cde49c96b46f5b5716dcda2590837 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 02:52:28 UTC) from {username='oryx', principal='oryx/127.12.138.254@KRBTEST.COM'} at 127.0.0.1:60306
I20260811 02:52:28.070814 22399 tablet_replica.cc:330] T a31cde49c96b46f5b5716dcda2590837 P 6c8a58fdbe944622b27ade12cd92a62e: stopping tablet replica
I20260811 02:52:28.070940 22399 raft_consensus.cc:2229] T a31cde49c96b46f5b5716dcda2590837 P 6c8a58fdbe944622b27ade12cd92a62e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 02:52:28.071044 22399 raft_consensus.cc:2258] T a31cde49c96b46f5b5716dcda2590837 P 6c8a58fdbe944622b27ade12cd92a62e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 02:52:28.071524 21915 tablet_service.cc:1514] Processing DeleteTablet for tablet a31cde49c96b46f5b5716dcda2590837 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 02:52:28 UTC) from {username='oryx', principal='oryx/127.12.138.254@KRBTEST.COM'} at 127.0.0.1:35364
I20260811 02:52:28.075539 22401 tablet_replica.cc:330] T a31cde49c96b46f5b5716dcda2590837 P 6937bfa1e57344d78f8cc3b5df278e87: stopping tablet replica
I20260811 02:52:28.075672 22401 raft_consensus.cc:2229] T a31cde49c96b46f5b5716dcda2590837 P 6937bfa1e57344d78f8cc3b5df278e87 [term 1 LEADER]: Raft consensus shutting down.
I20260811 02:52:28.075786 22399 ts_tablet_manager.cc:1905] T a31cde49c96b46f5b5716dcda2590837 P 6c8a58fdbe944622b27ade12cd92a62e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 02:52:28.075996 22401 raft_consensus.cc:2258] T a31cde49c96b46f5b5716dcda2590837 P 6937bfa1e57344d78f8cc3b5df278e87 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 02:52:28.076659 22401 ts_tablet_manager.cc:1905] T a31cde49c96b46f5b5716dcda2590837 P 6937bfa1e57344d78f8cc3b5df278e87: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 02:52:28.077299 22400 tablet_replica.cc:330] T a31cde49c96b46f5b5716dcda2590837 P 6a04a1a9c41e4f7fb9064c78468ba498: stopping tablet replica
I20260811 02:52:28.077427 22400 raft_consensus.cc:2229] T a31cde49c96b46f5b5716dcda2590837 P 6a04a1a9c41e4f7fb9064c78468ba498 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 02:52:28.077543 22400 raft_consensus.cc:2258] T a31cde49c96b46f5b5716dcda2590837 P 6a04a1a9c41e4f7fb9064c78468ba498 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 02:52:28.078203 22400 ts_tablet_manager.cc:1905] T a31cde49c96b46f5b5716dcda2590837 P 6a04a1a9c41e4f7fb9064c78468ba498: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 02:52:28.079391 22399 ts_tablet_manager.cc:1918] T a31cde49c96b46f5b5716dcda2590837 P 6c8a58fdbe944622b27ade12cd92a62e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 02:52:28.079525 22399 log.cc:1196] T a31cde49c96b46f5b5716dcda2590837 P 6c8a58fdbe944622b27ade12cd92a62e: Deleting WAL directory at /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786416707705494-12843-0/minicluster-data/ts-1/wal/wals/a31cde49c96b46f5b5716dcda2590837
I20260811 02:52:28.079664 22401 ts_tablet_manager.cc:1918] T a31cde49c96b46f5b5716dcda2590837 P 6937bfa1e57344d78f8cc3b5df278e87: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 02:52:28.079764 22401 log.cc:1196] T a31cde49c96b46f5b5716dcda2590837 P 6937bfa1e57344d78f8cc3b5df278e87: Deleting WAL directory at /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786416707705494-12843-0/minicluster-data/ts-0/wal/wals/a31cde49c96b46f5b5716dcda2590837
I20260811 02:52:28.080008 22399 ts_tablet_manager.cc:1939] T a31cde49c96b46f5b5716dcda2590837 P 6c8a58fdbe944622b27ade12cd92a62e: Deleting consensus metadata
I20260811 02:52:28.080106 22401 ts_tablet_manager.cc:1939] T a31cde49c96b46f5b5716dcda2590837 P 6937bfa1e57344d78f8cc3b5df278e87: Deleting consensus metadata
I20260811 02:52:28.080847 21746 catalog_manager.cc:4841] TS 6c8a58fdbe944622b27ade12cd92a62e (127.12.138.194:39121): tablet a31cde49c96b46f5b5716dcda2590837 (table test-table [id=6accdb9d6e434086808afd669a191c62]) successfully deleted
I20260811 02:52:28.081063 21745 catalog_manager.cc:4841] TS 6937bfa1e57344d78f8cc3b5df278e87 (127.12.138.193:34941): tablet a31cde49c96b46f5b5716dcda2590837 (table test-table [id=6accdb9d6e434086808afd669a191c62]) successfully deleted
I20260811 02:52:28.081175 22400 ts_tablet_manager.cc:1918] T a31cde49c96b46f5b5716dcda2590837 P 6a04a1a9c41e4f7fb9064c78468ba498: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 02:52:28.081266 22400 log.cc:1196] T a31cde49c96b46f5b5716dcda2590837 P 6a04a1a9c41e4f7fb9064c78468ba498: Deleting WAL directory at /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786416707705494-12843-0/minicluster-data/ts-2/wal/wals/a31cde49c96b46f5b5716dcda2590837
I20260811 02:52:28.081533 22400 ts_tablet_manager.cc:1939] T a31cde49c96b46f5b5716dcda2590837 P 6a04a1a9c41e4f7fb9064c78468ba498: Deleting consensus metadata
I20260811 02:52:28.082188 21744 catalog_manager.cc:4841] TS 6a04a1a9c41e4f7fb9064c78468ba498 (127.12.138.195:40285): tablet a31cde49c96b46f5b5716dcda2590837 (table test-table [id=6accdb9d6e434086808afd669a191c62]) successfully deleted
Aug 11 02:52:28 dist-test-slave-9gcw krb5kdc[21711](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416748, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 11 02:52:28 dist-test-slave-9gcw krb5kdc[21711](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.12.138.254@KRBTEST.COM, Server not found in Kerberos database
Aug 11 02:52:28 dist-test-slave-9gcw krb5kdc[21711](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.12.138.254@KRBTEST.COM, Server not found in Kerberos database
W20260811 02:52:28.113747 22380 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 02:52:28.101320 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36706 (local address 127.12.138.254:43149)
0811 02:52:28.101471 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0811 02:52:28.101477 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 02:52:28.101599 (+ 122us) server_negotiation.cc:407] Connection header received
0811 02:52:28.101754 (+ 155us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:52:28.101756 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:52:28.101820 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:52:28.101894 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:52:28.102795 (+ 901us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:52:28.103334 (+ 539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:52:28.104542 (+ 1208us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:52:28.104772 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:52:28.104864 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:52:28.113573 (+ 8709us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:36706: BlockingRecv error: recv got EOF from 127.0.0.1:36706 (error 108)
Metrics: {"server-negotiator.queue_time_us":46}
Aug 11 02:52:28 dist-test-slave-9gcw krb5kdc[21711](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416748, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.12.138.254@KRBTEST.COM
I20260811 02:52:28.124800 22380 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:52:28.115282 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36710 (local address 127.12.138.254:43149)
0811 02:52:28.115427 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0811 02:52:28.115432 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 02:52:28.115540 (+ 108us) server_negotiation.cc:407] Connection header received
0811 02:52:28.115680 (+ 140us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:52:28.115682 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:52:28.115729 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:52:28.115804 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:52:28.116520 (+ 716us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:52:28.117059 (+ 539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:52:28.118177 (+ 1118us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:52:28.118387 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:52:28.118468 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:52:28.121029 (+ 2561us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:52:28.121032 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:52:28.121034 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:52:28.121060 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:52:28.122850 (+ 1790us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:52:28.123324 (+ 474us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:52:28.123328 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:52:28.123330 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:52:28.123422 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:52:28.123749 (+ 327us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:52:28.123753 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:52:28.123754 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:52:28.123973 (+ 219us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:52:28.124118 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0811 02:52:28.124462 (+ 344us) server_negotiation.cc:299] Negotiation successful
0811 02:52:28.124611 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260811 02:52:28.127931 21760 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36710:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 02:52:28.128419 21760 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 02:52:28.133275 21915 tablet_service.cc:1467] Processing CreateTablet for tablet 1fcd7d6de36748b1b12fd2827c54044d (DEFAULT_TABLE table=test-table [id=9be28048f0d6402da0c93ec3b20e21f2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 02:52:28.133440 22073 tablet_service.cc:1467] Processing CreateTablet for tablet 1fcd7d6de36748b1b12fd2827c54044d (DEFAULT_TABLE table=test-table [id=9be28048f0d6402da0c93ec3b20e21f2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 02:52:28.133651 21915 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1fcd7d6de36748b1b12fd2827c54044d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 02:52:28.133785 22073 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1fcd7d6de36748b1b12fd2827c54044d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 02:52:28.135810 22340 tablet_bootstrap.cc:492] T 1fcd7d6de36748b1b12fd2827c54044d P 6c8a58fdbe944622b27ade12cd92a62e: Bootstrap starting.
I20260811 02:52:28.136053 22225 tablet_service.cc:1467] Processing CreateTablet for tablet 1fcd7d6de36748b1b12fd2827c54044d (DEFAULT_TABLE table=test-table [id=9be28048f0d6402da0c93ec3b20e21f2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 02:52:28.136340 22225 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1fcd7d6de36748b1b12fd2827c54044d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 02:52:28.136700 22340 tablet_bootstrap.cc:654] T 1fcd7d6de36748b1b12fd2827c54044d P 6c8a58fdbe944622b27ade12cd92a62e: Neither blocks nor log segments found. Creating new log.
I20260811 02:52:28.136970 22338 tablet_bootstrap.cc:492] T 1fcd7d6de36748b1b12fd2827c54044d P 6937bfa1e57344d78f8cc3b5df278e87: Bootstrap starting.
I20260811 02:52:28.137838 22338 tablet_bootstrap.cc:654] T 1fcd7d6de36748b1b12fd2827c54044d P 6937bfa1e57344d78f8cc3b5df278e87: Neither blocks nor log segments found. Creating new log.
I20260811 02:52:28.139014 22337 tablet_bootstrap.cc:492] T 1fcd7d6de36748b1b12fd2827c54044d P 6a04a1a9c41e4f7fb9064c78468ba498: Bootstrap starting.
I20260811 02:52:28.139497 22338 tablet_bootstrap.cc:492] T 1fcd7d6de36748b1b12fd2827c54044d P 6937bfa1e57344d78f8cc3b5df278e87: No bootstrap required, opened a new log
I20260811 02:52:28.139582 22338 ts_tablet_manager.cc:1397] T 1fcd7d6de36748b1b12fd2827c54044d P 6937bfa1e57344d78f8cc3b5df278e87: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260811 02:52:28.139645 22340 tablet_bootstrap.cc:492] T 1fcd7d6de36748b1b12fd2827c54044d P 6c8a58fdbe944622b27ade12cd92a62e: No bootstrap required, opened a new log
I20260811 02:52:28.139724 22340 ts_tablet_manager.cc:1397] T 1fcd7d6de36748b1b12fd2827c54044d P 6c8a58fdbe944622b27ade12cd92a62e: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.000s
I20260811 02:52:28.140009 22337 tablet_bootstrap.cc:654] T 1fcd7d6de36748b1b12fd2827c54044d P 6a04a1a9c41e4f7fb9064c78468ba498: Neither blocks nor log segments found. Creating new log.
I20260811 02:52:28.140022 22338 raft_consensus.cc:348] T 1fcd7d6de36748b1b12fd2827c54044d P 6937bfa1e57344d78f8cc3b5df278e87 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6937bfa1e57344d78f8cc3b5df278e87" member_type: VOTER last_known_addr { host: "127.12.138.193" port: 34941 } } peers { permanent_uuid: "6c8a58fdbe944622b27ade12cd92a62e" member_type: VOTER last_known_addr { host: "127.12.138.194" port: 39121 } } peers { permanent_uuid: "6a04a1a9c41e4f7fb9064c78468ba498" member_type: VOTER last_known_addr { host: "127.12.138.195" port: 40285 } }
I20260811 02:52:28.140153 22338 raft_consensus.cc:374] T 1fcd7d6de36748b1b12fd2827c54044d P 6937bfa1e57344d78f8cc3b5df278e87 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 02:52:28.140188 22338 raft_consensus.cc:729] T 1fcd7d6de36748b1b12fd2827c54044d P 6937bfa1e57344d78f8cc3b5df278e87 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6937bfa1e57344d78f8cc3b5df278e87, State: Initialized, Role: FOLLOWER
I20260811 02:52:28.140168 22340 raft_consensus.cc:348] T 1fcd7d6de36748b1b12fd2827c54044d P 6c8a58fdbe944622b27ade12cd92a62e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6937bfa1e57344d78f8cc3b5df278e87" member_type: VOTER last_known_addr { host: "127.12.138.193" port: 34941 } } peers { permanent_uuid: "6c8a58fdbe944622b27ade12cd92a62e" member_type: VOTER last_known_addr { host: "127.12.138.194" port: 39121 } } peers { permanent_uuid: "6a04a1a9c41e4f7fb9064c78468ba498" member_type: VOTER last_known_addr { host: "127.12.138.195" port: 40285 } }
I20260811 02:52:28.140295 22340 raft_consensus.cc:374] T 1fcd7d6de36748b1b12fd2827c54044d P 6c8a58fdbe944622b27ade12cd92a62e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 02:52:28.140340 22340 raft_consensus.cc:729] T 1fcd7d6de36748b1b12fd2827c54044d P 6c8a58fdbe944622b27ade12cd92a62e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6c8a58fdbe944622b27ade12cd92a62e, State: Initialized, Role: FOLLOWER
I20260811 02:52:28.140319 22338 consensus_queue.cc:260] T 1fcd7d6de36748b1b12fd2827c54044d P 6937bfa1e57344d78f8cc3b5df278e87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6937bfa1e57344d78f8cc3b5df278e87" member_type: VOTER last_known_addr { host: "127.12.138.193" port: 34941 } } peers { permanent_uuid: "6c8a58fdbe944622b27ade12cd92a62e" member_type: VOTER last_known_addr { host: "127.12.138.194" port: 39121 } } peers { permanent_uuid: "6a04a1a9c41e4f7fb9064c78468ba498" member_type: VOTER last_known_addr { host: "127.12.138.195" port: 40285 } }
I20260811 02:52:28.140487 22340 consensus_queue.cc:260] T 1fcd7d6de36748b1b12fd2827c54044d P 6c8a58fdbe944622b27ade12cd92a62e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6937bfa1e57344d78f8cc3b5df278e87" member_type: VOTER last_known_addr { host: "127.12.138.193" port: 34941 } } peers { permanent_uuid: "6c8a58fdbe944622b27ade12cd92a62e" member_type: VOTER last_known_addr { host: "127.12.138.194" port: 39121 } } peers { permanent_uuid: "6a04a1a9c41e4f7fb9064c78468ba498" member_type: VOTER last_known_addr { host: "127.12.138.195" port: 40285 } }
I20260811 02:52:28.140604 22338 ts_tablet_manager.cc:1428] T 1fcd7d6de36748b1b12fd2827c54044d P 6937bfa1e57344d78f8cc3b5df278e87: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260811 02:52:28.140822 22340 ts_tablet_manager.cc:1428] T 1fcd7d6de36748b1b12fd2827c54044d P 6c8a58fdbe944622b27ade12cd92a62e: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260811 02:52:28.142885 22337 tablet_bootstrap.cc:492] T 1fcd7d6de36748b1b12fd2827c54044d P 6a04a1a9c41e4f7fb9064c78468ba498: No bootstrap required, opened a new log
I20260811 02:52:28.142992 22337 ts_tablet_manager.cc:1397] T 1fcd7d6de36748b1b12fd2827c54044d P 6a04a1a9c41e4f7fb9064c78468ba498: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.001s
I20260811 02:52:28.143517 22337 raft_consensus.cc:348] T 1fcd7d6de36748b1b12fd2827c54044d P 6a04a1a9c41e4f7fb9064c78468ba498 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6937bfa1e57344d78f8cc3b5df278e87" member_type: VOTER last_known_addr { host: "127.12.138.193" port: 34941 } } peers { permanent_uuid: "6c8a58fdbe944622b27ade12cd92a62e" member_type: VOTER last_known_addr { host: "127.12.138.194" port: 39121 } } peers { permanent_uuid: "6a04a1a9c41e4f7fb9064c78468ba498" member_type: VOTER last_known_addr { host: "127.12.138.195" port: 40285 } }
I20260811 02:52:28.143647 22337 raft_consensus.cc:374] T 1fcd7d6de36748b1b12fd2827c54044d P 6a04a1a9c41e4f7fb9064c78468ba498 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 02:52:28.143692 22337 raft_consensus.cc:729] T 1fcd7d6de36748b1b12fd2827c54044d P 6a04a1a9c41e4f7fb9064c78468ba498 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6a04a1a9c41e4f7fb9064c78468ba498, State: Initialized, Role: FOLLOWER
I20260811 02:52:28.143831 22337 consensus_queue.cc:260] T 1fcd7d6de36748b1b12fd2827c54044d P 6a04a1a9c41e4f7fb9064c78468ba498 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6937bfa1e57344d78f8cc3b5df278e87" member_type: VOTER last_known_addr { host: "127.12.138.193" port: 34941 } } peers { permanent_uuid: "6c8a58fdbe944622b27ade12cd92a62e" member_type: VOTER last_known_addr { host: "127.12.138.194" port: 39121 } } peers { permanent_uuid: "6a04a1a9c41e4f7fb9064c78468ba498" member_type: VOTER last_known_addr { host: "127.12.138.195" port: 40285 } }
I20260811 02:52:28.145437 22337 ts_tablet_manager.cc:1428] T 1fcd7d6de36748b1b12fd2827c54044d P 6a04a1a9c41e4f7fb9064c78468ba498: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20260811 02:52:28.170315 22395 raft_consensus.cc:482] T 1fcd7d6de36748b1b12fd2827c54044d P 6937bfa1e57344d78f8cc3b5df278e87 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 02:52:28.170472 22395 raft_consensus.cc:504] T 1fcd7d6de36748b1b12fd2827c54044d P 6937bfa1e57344d78f8cc3b5df278e87 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6937bfa1e57344d78f8cc3b5df278e87" member_type: VOTER last_known_addr { host: "127.12.138.193" port: 34941 } } peers { permanent_uuid: "6c8a58fdbe944622b27ade12cd92a62e" member_type: VOTER last_known_addr { host: "127.12.138.194" port: 39121 } } peers { permanent_uuid: "6a04a1a9c41e4f7fb9064c78468ba498" member_type: VOTER last_known_addr { host: "127.12.138.195" port: 40285 } }
I20260811 02:52:28.170995 22395 leader_election.cc:288] T 1fcd7d6de36748b1b12fd2827c54044d P 6937bfa1e57344d78f8cc3b5df278e87 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6c8a58fdbe944622b27ade12cd92a62e (127.12.138.194:39121), 6a04a1a9c41e4f7fb9064c78468ba498 (127.12.138.195:40285)
I20260811 02:52:28.171463 22249 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1fcd7d6de36748b1b12fd2827c54044d" candidate_uuid: "6937bfa1e57344d78f8cc3b5df278e87" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6a04a1a9c41e4f7fb9064c78468ba498" is_pre_election: true
I20260811 02:52:28.171460 22093 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1fcd7d6de36748b1b12fd2827c54044d" candidate_uuid: "6937bfa1e57344d78f8cc3b5df278e87" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6c8a58fdbe944622b27ade12cd92a62e" is_pre_election: true
I20260811 02:52:28.171619 22093 raft_consensus.cc:2454] T 1fcd7d6de36748b1b12fd2827c54044d P 6c8a58fdbe944622b27ade12cd92a62e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6937bfa1e57344d78f8cc3b5df278e87 in term 0.
I20260811 02:52:28.171640 22249 raft_consensus.cc:2454] T 1fcd7d6de36748b1b12fd2827c54044d P 6a04a1a9c41e4f7fb9064c78468ba498 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6937bfa1e57344d78f8cc3b5df278e87 in term 0.
I20260811 02:52:28.171972 21862 leader_election.cc:302] T 1fcd7d6de36748b1b12fd2827c54044d P 6937bfa1e57344d78f8cc3b5df278e87 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6937bfa1e57344d78f8cc3b5df278e87, 6c8a58fdbe944622b27ade12cd92a62e; no voters:
I20260811 02:52:28.172168 22395 raft_consensus.cc:2781] T 1fcd7d6de36748b1b12fd2827c54044d P 6937bfa1e57344d78f8cc3b5df278e87 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 02:52:28.172261 22395 raft_consensus.cc:482] T 1fcd7d6de36748b1b12fd2827c54044d P 6937bfa1e57344d78f8cc3b5df278e87 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 02:52:28.172322 22395 raft_consensus.cc:3037] T 1fcd7d6de36748b1b12fd2827c54044d P 6937bfa1e57344d78f8cc3b5df278e87 [term 0 FOLLOWER]: Advancing to term 1
I20260811 02:52:28.173508 22395 raft_consensus.cc:504] T 1fcd7d6de36748b1b12fd2827c54044d P 6937bfa1e57344d78f8cc3b5df278e87 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6937bfa1e57344d78f8cc3b5df278e87" member_type: VOTER last_known_addr { host: "127.12.138.193" port: 34941 } } peers { permanent_uuid: "6c8a58fdbe944622b27ade12cd92a62e" member_type: VOTER last_known_addr { host: "127.12.138.194" port: 39121 } } peers { permanent_uuid: "6a04a1a9c41e4f7fb9064c78468ba498" member_type: VOTER last_known_addr { host: "127.12.138.195" port: 40285 } }
I20260811 02:52:28.173892 22395 leader_election.cc:288] T 1fcd7d6de36748b1b12fd2827c54044d P 6937bfa1e57344d78f8cc3b5df278e87 [CANDIDATE]: Term 1 election: Requested vote from peers 6c8a58fdbe944622b27ade12cd92a62e (127.12.138.194:39121), 6a04a1a9c41e4f7fb9064c78468ba498 (127.12.138.195:40285)
I20260811 02:52:28.174232 22093 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1fcd7d6de36748b1b12fd2827c54044d" candidate_uuid: "6937bfa1e57344d78f8cc3b5df278e87" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6c8a58fdbe944622b27ade12cd92a62e"
I20260811 02:52:28.174284 22249 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1fcd7d6de36748b1b12fd2827c54044d" candidate_uuid: "6937bfa1e57344d78f8cc3b5df278e87" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6a04a1a9c41e4f7fb9064c78468ba498"
I20260811 02:52:28.174371 22093 raft_consensus.cc:3037] T 1fcd7d6de36748b1b12fd2827c54044d P 6c8a58fdbe944622b27ade12cd92a62e [term 0 FOLLOWER]: Advancing to term 1
I20260811 02:52:28.174413 22249 raft_consensus.cc:3037] T 1fcd7d6de36748b1b12fd2827c54044d P 6a04a1a9c41e4f7fb9064c78468ba498 [term 0 FOLLOWER]: Advancing to term 1
I20260811 02:52:28.175350 22249 raft_consensus.cc:2454] T 1fcd7d6de36748b1b12fd2827c54044d P 6a04a1a9c41e4f7fb9064c78468ba498 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6937bfa1e57344d78f8cc3b5df278e87 in term 1.
I20260811 02:52:28.175643 21860 leader_election.cc:302] T 1fcd7d6de36748b1b12fd2827c54044d P 6937bfa1e57344d78f8cc3b5df278e87 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6937bfa1e57344d78f8cc3b5df278e87, 6a04a1a9c41e4f7fb9064c78468ba498; no voters:
I20260811 02:52:28.175647 22093 raft_consensus.cc:2454] T 1fcd7d6de36748b1b12fd2827c54044d P 6c8a58fdbe944622b27ade12cd92a62e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6937bfa1e57344d78f8cc3b5df278e87 in term 1.
I20260811 02:52:28.175863 22395 raft_consensus.cc:2781] T 1fcd7d6de36748b1b12fd2827c54044d P 6937bfa1e57344d78f8cc3b5df278e87 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 02:52:28.175954 22395 raft_consensus.cc:686] T 1fcd7d6de36748b1b12fd2827c54044d P 6937bfa1e57344d78f8cc3b5df278e87 [term 1 LEADER]: Becoming Leader. State: Replica: 6937bfa1e57344d78f8cc3b5df278e87, State: Running, Role: LEADER
I20260811 02:52:28.176095 22395 consensus_queue.cc:237] T 1fcd7d6de36748b1b12fd2827c54044d P 6937bfa1e57344d78f8cc3b5df278e87 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6937bfa1e57344d78f8cc3b5df278e87" member_type: VOTER last_known_addr { host: "127.12.138.193" port: 34941 } } peers { permanent_uuid: "6c8a58fdbe944622b27ade12cd92a62e" member_type: VOTER last_known_addr { host: "127.12.138.194" port: 39121 } } peers { permanent_uuid: "6a04a1a9c41e4f7fb9064c78468ba498" member_type: VOTER last_known_addr { host: "127.12.138.195" port: 40285 } }
I20260811 02:52:28.178037 21760 catalog_manager.cc:5495] T 1fcd7d6de36748b1b12fd2827c54044d P 6937bfa1e57344d78f8cc3b5df278e87 reported cstate change: term changed from 0 to 1, leader changed from <none> to 6937bfa1e57344d78f8cc3b5df278e87 (127.12.138.193). New cstate: current_term: 1 leader_uuid: "6937bfa1e57344d78f8cc3b5df278e87" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6937bfa1e57344d78f8cc3b5df278e87" member_type: VOTER last_known_addr { host: "127.12.138.193" port: 34941 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6c8a58fdbe944622b27ade12cd92a62e" member_type: VOTER last_known_addr { host: "127.12.138.194" port: 39121 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6a04a1a9c41e4f7fb9064c78468ba498" member_type: VOTER last_known_addr { host: "127.12.138.195" port: 40285 } health_report { overall_health: UNKNOWN } } }
I20260811 02:52:28.196977 22389 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:52:28.191853 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35406 (local address 127.12.138.193:34941)
0811 02:52:28.192189 (+ 336us) server_negotiation.cc:206] Beginning negotiation
0811 02:52:28.192193 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 02:52:28.192203 (+ 10us) server_negotiation.cc:407] Connection header received
0811 02:52:28.192268 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:52:28.192274 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:52:28.192328 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:52:28.192405 (+ 77us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 02:52:28.192729 (+ 324us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:52:28.194329 (+ 1600us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:52:28.195530 (+ 1201us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:52:28.195895 (+ 365us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:52:28.196002 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:52:28.196379 (+ 377us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 02:52:28.196482 (+ 103us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 02:52:28.196573 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0811 02:52:28.196718 (+ 145us) server_negotiation.cc:299] Negotiation successful
0811 02:52:28.196803 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":263}
I20260811 02:52:28.202893 22093 raft_consensus.cc:1261] T 1fcd7d6de36748b1b12fd2827c54044d P 6c8a58fdbe944622b27ade12cd92a62e [term 1 FOLLOWER]: Refusing update from remote peer 6937bfa1e57344d78f8cc3b5df278e87: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 02:52:28.204104 22249 raft_consensus.cc:1261] T 1fcd7d6de36748b1b12fd2827c54044d P 6a04a1a9c41e4f7fb9064c78468ba498 [term 1 FOLLOWER]: Refusing update from remote peer 6937bfa1e57344d78f8cc3b5df278e87: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 02:52:28.204511 22372 consensus_queue.cc:1035] T 1fcd7d6de36748b1b12fd2827c54044d P 6937bfa1e57344d78f8cc3b5df278e87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c8a58fdbe944622b27ade12cd92a62e" member_type: VOTER last_known_addr { host: "127.12.138.194" port: 39121 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 02:52:28.205350 22372 consensus_queue.cc:1035] T 1fcd7d6de36748b1b12fd2827c54044d P 6937bfa1e57344d78f8cc3b5df278e87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a04a1a9c41e4f7fb9064c78468ba498" member_type: VOTER last_known_addr { host: "127.12.138.195" port: 40285 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 02:52:28.218933 21760 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36710:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 02:52:28.220543 21760 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36710:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 02:52:28.223626 21760 catalog_manager.cc:5782] T 00000000000000000000000000000000 P acf0fa3758a14749a4a0ec3e106b3dd5: Sending DeleteTablet for 3 replicas of tablet 1fcd7d6de36748b1b12fd2827c54044d
I20260811 02:52:28.224363 21915 tablet_service.cc:1514] Processing DeleteTablet for tablet 1fcd7d6de36748b1b12fd2827c54044d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 02:52:28 UTC) from {username='oryx', principal='oryx/127.12.138.254@KRBTEST.COM'} at 127.0.0.1:35364
I20260811 02:52:28.224411 22073 tablet_service.cc:1514] Processing DeleteTablet for tablet 1fcd7d6de36748b1b12fd2827c54044d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 02:52:28 UTC) from {username='oryx', principal='oryx/127.12.138.254@KRBTEST.COM'} at 127.0.0.1:40316
I20260811 02:52:28.224606 22401 tablet_replica.cc:330] T 1fcd7d6de36748b1b12fd2827c54044d P 6937bfa1e57344d78f8cc3b5df278e87: stopping tablet replica
I20260811 02:52:28.224666 22399 tablet_replica.cc:330] T 1fcd7d6de36748b1b12fd2827c54044d P 6c8a58fdbe944622b27ade12cd92a62e: stopping tablet replica
I20260811 02:52:28.224759 22401 raft_consensus.cc:2229] T 1fcd7d6de36748b1b12fd2827c54044d P 6937bfa1e57344d78f8cc3b5df278e87 [term 1 LEADER]: Raft consensus shutting down.
I20260811 02:52:28.224812 22399 raft_consensus.cc:2229] T 1fcd7d6de36748b1b12fd2827c54044d P 6c8a58fdbe944622b27ade12cd92a62e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 02:52:28.224946 22399 raft_consensus.cc:2258] T 1fcd7d6de36748b1b12fd2827c54044d P 6c8a58fdbe944622b27ade12cd92a62e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 02:52:28.224989 22401 raft_consensus.cc:2258] T 1fcd7d6de36748b1b12fd2827c54044d P 6937bfa1e57344d78f8cc3b5df278e87 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 02:52:28.225788 22401 ts_tablet_manager.cc:1905] T 1fcd7d6de36748b1b12fd2827c54044d P 6937bfa1e57344d78f8cc3b5df278e87: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 02:52:28.226008 22225 tablet_service.cc:1514] Processing DeleteTablet for tablet 1fcd7d6de36748b1b12fd2827c54044d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 02:52:28 UTC) from {username='oryx', principal='oryx/127.12.138.254@KRBTEST.COM'} at 127.0.0.1:60306
I20260811 02:52:28.226227 22400 tablet_replica.cc:330] T 1fcd7d6de36748b1b12fd2827c54044d P 6a04a1a9c41e4f7fb9064c78468ba498: stopping tablet replica
I20260811 02:52:28.226356 22400 raft_consensus.cc:2229] T 1fcd7d6de36748b1b12fd2827c54044d P 6a04a1a9c41e4f7fb9064c78468ba498 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 02:52:28.226505 22400 raft_consensus.cc:2258] T 1fcd7d6de36748b1b12fd2827c54044d P 6a04a1a9c41e4f7fb9064c78468ba498 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 02:52:28.227035 22399 ts_tablet_manager.cc:1905] T 1fcd7d6de36748b1b12fd2827c54044d P 6c8a58fdbe944622b27ade12cd92a62e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 02:52:28.228816 22401 ts_tablet_manager.cc:1918] T 1fcd7d6de36748b1b12fd2827c54044d P 6937bfa1e57344d78f8cc3b5df278e87: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 02:52:28.228914 22401 log.cc:1196] T 1fcd7d6de36748b1b12fd2827c54044d P 6937bfa1e57344d78f8cc3b5df278e87: Deleting WAL directory at /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786416707705494-12843-0/minicluster-data/ts-0/wal/wals/1fcd7d6de36748b1b12fd2827c54044d
I20260811 02:52:28.229590 22401 ts_tablet_manager.cc:1939] T 1fcd7d6de36748b1b12fd2827c54044d P 6937bfa1e57344d78f8cc3b5df278e87: Deleting consensus metadata
I20260811 02:52:28.230513 21745 catalog_manager.cc:4841] TS 6937bfa1e57344d78f8cc3b5df278e87 (127.12.138.193:34941): tablet 1fcd7d6de36748b1b12fd2827c54044d (table test-table [id=9be28048f0d6402da0c93ec3b20e21f2]) successfully deleted
I20260811 02:52:28.230615 22399 ts_tablet_manager.cc:1918] T 1fcd7d6de36748b1b12fd2827c54044d P 6c8a58fdbe944622b27ade12cd92a62e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 02:52:28.230703 22399 log.cc:1196] T 1fcd7d6de36748b1b12fd2827c54044d P 6c8a58fdbe944622b27ade12cd92a62e: Deleting WAL directory at /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786416707705494-12843-0/minicluster-data/ts-1/wal/wals/1fcd7d6de36748b1b12fd2827c54044d
I20260811 02:52:28.231029 22399 ts_tablet_manager.cc:1939] T 1fcd7d6de36748b1b12fd2827c54044d P 6c8a58fdbe944622b27ade12cd92a62e: Deleting consensus metadata
I20260811 02:52:28.231698 22400 ts_tablet_manager.cc:1905] T 1fcd7d6de36748b1b12fd2827c54044d P 6a04a1a9c41e4f7fb9064c78468ba498: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 02:52:28.231805 21746 catalog_manager.cc:4841] TS 6c8a58fdbe944622b27ade12cd92a62e (127.12.138.194:39121): tablet 1fcd7d6de36748b1b12fd2827c54044d (table test-table [id=9be28048f0d6402da0c93ec3b20e21f2]) successfully deleted
I20260811 02:52:28.232182 21760 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36710:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 02:52:28.232678 21760 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 02:52:28.234449 22400 ts_tablet_manager.cc:1918] T 1fcd7d6de36748b1b12fd2827c54044d P 6a04a1a9c41e4f7fb9064c78468ba498: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 02:52:28.234608 22400 log.cc:1196] T 1fcd7d6de36748b1b12fd2827c54044d P 6a04a1a9c41e4f7fb9064c78468ba498: Deleting WAL directory at /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786416707705494-12843-0/minicluster-data/ts-2/wal/wals/1fcd7d6de36748b1b12fd2827c54044d
I20260811 02:52:28.234927 22400 ts_tablet_manager.cc:1939] T 1fcd7d6de36748b1b12fd2827c54044d P 6a04a1a9c41e4f7fb9064c78468ba498: Deleting consensus metadata
I20260811 02:52:28.235755 21744 catalog_manager.cc:4841] TS 6a04a1a9c41e4f7fb9064c78468ba498 (127.12.138.195:40285): tablet 1fcd7d6de36748b1b12fd2827c54044d (table test-table [id=9be28048f0d6402da0c93ec3b20e21f2]) successfully deleted
I20260811 02:52:28.239565 22225 tablet_service.cc:1467] Processing CreateTablet for tablet 2a1eee69ea8549cbbb34c746f0fd69e2 (DEFAULT_TABLE table=test-table [id=8ee467ecab784d0a93757a92c0929fd7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 02:52:28.239650 22073 tablet_service.cc:1467] Processing CreateTablet for tablet 2a1eee69ea8549cbbb34c746f0fd69e2 (DEFAULT_TABLE table=test-table [id=8ee467ecab784d0a93757a92c0929fd7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 02:52:28.239892 22225 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2a1eee69ea8549cbbb34c746f0fd69e2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 02:52:28.239972 22073 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2a1eee69ea8549cbbb34c746f0fd69e2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 02:52:28.242619 22337 tablet_bootstrap.cc:492] T 2a1eee69ea8549cbbb34c746f0fd69e2 P 6a04a1a9c41e4f7fb9064c78468ba498: Bootstrap starting.
I20260811 02:52:28.242861 22340 tablet_bootstrap.cc:492] T 2a1eee69ea8549cbbb34c746f0fd69e2 P 6c8a58fdbe944622b27ade12cd92a62e: Bootstrap starting.
I20260811 02:52:28.243741 22337 tablet_bootstrap.cc:654] T 2a1eee69ea8549cbbb34c746f0fd69e2 P 6a04a1a9c41e4f7fb9064c78468ba498: Neither blocks nor log segments found. Creating new log.
I20260811 02:52:28.243770 22340 tablet_bootstrap.cc:654] T 2a1eee69ea8549cbbb34c746f0fd69e2 P 6c8a58fdbe944622b27ade12cd92a62e: Neither blocks nor log segments found. Creating new log.
I20260811 02:52:28.244675 21915 tablet_service.cc:1467] Processing CreateTablet for tablet 2a1eee69ea8549cbbb34c746f0fd69e2 (DEFAULT_TABLE table=test-table [id=8ee467ecab784d0a93757a92c0929fd7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 02:52:28.244882 22340 tablet_bootstrap.cc:492] T 2a1eee69ea8549cbbb34c746f0fd69e2 P 6c8a58fdbe944622b27ade12cd92a62e: No bootstrap required, opened a new log
I20260811 02:52:28.244951 22340 ts_tablet_manager.cc:1397] T 2a1eee69ea8549cbbb34c746f0fd69e2 P 6c8a58fdbe944622b27ade12cd92a62e: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260811 02:52:28.244978 21915 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2a1eee69ea8549cbbb34c746f0fd69e2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 02:52:28.245427 22340 raft_consensus.cc:348] T 2a1eee69ea8549cbbb34c746f0fd69e2 P 6c8a58fdbe944622b27ade12cd92a62e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6937bfa1e57344d78f8cc3b5df278e87" member_type: VOTER last_known_addr { host: "127.12.138.193" port: 34941 } } peers { permanent_uuid: "6c8a58fdbe944622b27ade12cd92a62e" member_type: VOTER last_known_addr { host: "127.12.138.194" port: 39121 } } peers { permanent_uuid: "6a04a1a9c41e4f7fb9064c78468ba498" member_type: VOTER last_known_addr { host: "127.12.138.195" port: 40285 } }
I20260811 02:52:28.245563 22340 raft_consensus.cc:374] T 2a1eee69ea8549cbbb34c746f0fd69e2 P 6c8a58fdbe944622b27ade12cd92a62e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 02:52:28.245602 22340 raft_consensus.cc:729] T 2a1eee69ea8549cbbb34c746f0fd69e2 P 6c8a58fdbe944622b27ade12cd92a62e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6c8a58fdbe944622b27ade12cd92a62e, State: Initialized, Role: FOLLOWER
I20260811 02:52:28.245746 22340 consensus_queue.cc:260] T 2a1eee69ea8549cbbb34c746f0fd69e2 P 6c8a58fdbe944622b27ade12cd92a62e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6937bfa1e57344d78f8cc3b5df278e87" member_type: VOTER last_known_addr { host: "127.12.138.193" port: 34941 } } peers { permanent_uuid: "6c8a58fdbe944622b27ade12cd92a62e" member_type: VOTER last_known_addr { host: "127.12.138.194" port: 39121 } } peers { permanent_uuid: "6a04a1a9c41e4f7fb9064c78468ba498" member_type: VOTER last_known_addr { host: "127.12.138.195" port: 40285 } }
I20260811 02:52:28.246040 22340 ts_tablet_manager.cc:1428] T 2a1eee69ea8549cbbb34c746f0fd69e2 P 6c8a58fdbe944622b27ade12cd92a62e: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260811 02:52:28.248379 22337 tablet_bootstrap.cc:492] T 2a1eee69ea8549cbbb34c746f0fd69e2 P 6a04a1a9c41e4f7fb9064c78468ba498: No bootstrap required, opened a new log
I20260811 02:52:28.248481 22337 ts_tablet_manager.cc:1397] T 2a1eee69ea8549cbbb34c746f0fd69e2 P 6a04a1a9c41e4f7fb9064c78468ba498: Time spent bootstrapping tablet: real 0.006s user 0.002s sys 0.000s
I20260811 02:52:28.249023 22337 raft_consensus.cc:348] T 2a1eee69ea8549cbbb34c746f0fd69e2 P 6a04a1a9c41e4f7fb9064c78468ba498 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6937bfa1e57344d78f8cc3b5df278e87" member_type: VOTER last_known_addr { host: "127.12.138.193" port: 34941 } } peers { permanent_uuid: "6c8a58fdbe944622b27ade12cd92a62e" member_type: VOTER last_known_addr { host: "127.12.138.194" port: 39121 } } peers { permanent_uuid: "6a04a1a9c41e4f7fb9064c78468ba498" member_type: VOTER last_known_addr { host: "127.12.138.195" port: 40285 } }
I20260811 02:52:28.249174 22337 raft_consensus.cc:374] T 2a1eee69ea8549cbbb34c746f0fd69e2 P 6a04a1a9c41e4f7fb9064c78468ba498 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 02:52:28.249223 22337 raft_consensus.cc:729] T 2a1eee69ea8549cbbb34c746f0fd69e2 P 6a04a1a9c41e4f7fb9064c78468ba498 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6a04a1a9c41e4f7fb9064c78468ba498, State: Initialized, Role: FOLLOWER
I20260811 02:52:28.249827 22337 consensus_queue.cc:260] T 2a1eee69ea8549cbbb34c746f0fd69e2 P 6a04a1a9c41e4f7fb9064c78468ba498 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6937bfa1e57344d78f8cc3b5df278e87" member_type: VOTER last_known_addr { host: "127.12.138.193" port: 34941 } } peers { permanent_uuid: "6c8a58fdbe944622b27ade12cd92a62e" member_type: VOTER last_known_addr { host: "127.12.138.194" port: 39121 } } peers { permanent_uuid: "6a04a1a9c41e4f7fb9064c78468ba498" member_type: VOTER last_known_addr { host: "127.12.138.195" port: 40285 } }
I20260811 02:52:28.250178 22337 ts_tablet_manager.cc:1428] T 2a1eee69ea8549cbbb34c746f0fd69e2 P 6a04a1a9c41e4f7fb9064c78468ba498: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20260811 02:52:28.250497 22338 tablet_bootstrap.cc:492] T 2a1eee69ea8549cbbb34c746f0fd69e2 P 6937bfa1e57344d78f8cc3b5df278e87: Bootstrap starting.
I20260811 02:52:28.251463 22338 tablet_bootstrap.cc:654] T 2a1eee69ea8549cbbb34c746f0fd69e2 P 6937bfa1e57344d78f8cc3b5df278e87: Neither blocks nor log segments found. Creating new log.
I20260811 02:52:28.253342 22338 tablet_bootstrap.cc:492] T 2a1eee69ea8549cbbb34c746f0fd69e2 P 6937bfa1e57344d78f8cc3b5df278e87: No bootstrap required, opened a new log
I20260811 02:52:28.253445 22338 ts_tablet_manager.cc:1397] T 2a1eee69ea8549cbbb34c746f0fd69e2 P 6937bfa1e57344d78f8cc3b5df278e87: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260811 02:52:28.253950 22338 raft_consensus.cc:348] T 2a1eee69ea8549cbbb34c746f0fd69e2 P 6937bfa1e57344d78f8cc3b5df278e87 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6937bfa1e57344d78f8cc3b5df278e87" member_type: VOTER last_known_addr { host: "127.12.138.193" port: 34941 } } peers { permanent_uuid: "6c8a58fdbe944622b27ade12cd92a62e" member_type: VOTER last_known_addr { host: "127.12.138.194" port: 39121 } } peers { permanent_uuid: "6a04a1a9c41e4f7fb9064c78468ba498" member_type: VOTER last_known_addr { host: "127.12.138.195" port: 40285 } }
I20260811 02:52:28.254092 22338 raft_consensus.cc:374] T 2a1eee69ea8549cbbb34c746f0fd69e2 P 6937bfa1e57344d78f8cc3b5df278e87 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 02:52:28.254132 22338 raft_consensus.cc:729] T 2a1eee69ea8549cbbb34c746f0fd69e2 P 6937bfa1e57344d78f8cc3b5df278e87 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6937bfa1e57344d78f8cc3b5df278e87, State: Initialized, Role: FOLLOWER
I20260811 02:52:28.254273 22338 consensus_queue.cc:260] T 2a1eee69ea8549cbbb34c746f0fd69e2 P 6937bfa1e57344d78f8cc3b5df278e87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6937bfa1e57344d78f8cc3b5df278e87" member_type: VOTER last_known_addr { host: "127.12.138.193" port: 34941 } } peers { permanent_uuid: "6c8a58fdbe944622b27ade12cd92a62e" member_type: VOTER last_known_addr { host: "127.12.138.194" port: 39121 } } peers { permanent_uuid: "6a04a1a9c41e4f7fb9064c78468ba498" member_type: VOTER last_known_addr { host: "127.12.138.195" port: 40285 } }
I20260811 02:52:28.256263 22338 ts_tablet_manager.cc:1428] T 2a1eee69ea8549cbbb34c746f0fd69e2 P 6937bfa1e57344d78f8cc3b5df278e87: Time spent starting tablet: real 0.003s user 0.001s sys 0.000s
I20260811 02:52:28.276589 22345 raft_consensus.cc:482] T 2a1eee69ea8549cbbb34c746f0fd69e2 P 6c8a58fdbe944622b27ade12cd92a62e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 02:52:28.276736 22345 raft_consensus.cc:504] T 2a1eee69ea8549cbbb34c746f0fd69e2 P 6c8a58fdbe944622b27ade12cd92a62e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6937bfa1e57344d78f8cc3b5df278e87" member_type: VOTER last_known_addr { host: "127.12.138.193" port: 34941 } } peers { permanent_uuid: "6c8a58fdbe944622b27ade12cd92a62e" member_type: VOTER last_known_addr { host: "127.12.138.194" port: 39121 } } peers { permanent_uuid: "6a04a1a9c41e4f7fb9064c78468ba498" member_type: VOTER last_known_addr { host: "127.12.138.195" port: 40285 } }
I20260811 02:52:28.277333 22345 leader_election.cc:288] T 2a1eee69ea8549cbbb34c746f0fd69e2 P 6c8a58fdbe944622b27ade12cd92a62e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6937bfa1e57344d78f8cc3b5df278e87 (127.12.138.193:34941), 6a04a1a9c41e4f7fb9064c78468ba498 (127.12.138.195:40285)
I20260811 02:52:28.277786 21937 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2a1eee69ea8549cbbb34c746f0fd69e2" candidate_uuid: "6c8a58fdbe944622b27ade12cd92a62e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6937bfa1e57344d78f8cc3b5df278e87" is_pre_election: true
I20260811 02:52:28.277956 21937 raft_consensus.cc:2454] T 2a1eee69ea8549cbbb34c746f0fd69e2 P 6937bfa1e57344d78f8cc3b5df278e87 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c8a58fdbe944622b27ade12cd92a62e in term 0.
I20260811 02:52:28.278275 22249 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2a1eee69ea8549cbbb34c746f0fd69e2" candidate_uuid: "6c8a58fdbe944622b27ade12cd92a62e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6a04a1a9c41e4f7fb9064c78468ba498" is_pre_election: true
I20260811 02:52:28.278301 22018 leader_election.cc:302] T 2a1eee69ea8549cbbb34c746f0fd69e2 P 6c8a58fdbe944622b27ade12cd92a62e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6937bfa1e57344d78f8cc3b5df278e87, 6c8a58fdbe944622b27ade12cd92a62e; no voters:
I20260811 02:52:28.278443 22249 raft_consensus.cc:2454] T 2a1eee69ea8549cbbb34c746f0fd69e2 P 6a04a1a9c41e4f7fb9064c78468ba498 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c8a58fdbe944622b27ade12cd92a62e in term 0.
I20260811 02:52:28.278505 22345 raft_consensus.cc:2781] T 2a1eee69ea8549cbbb34c746f0fd69e2 P 6c8a58fdbe944622b27ade12cd92a62e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 02:52:28.278558 22345 raft_consensus.cc:482] T 2a1eee69ea8549cbbb34c746f0fd69e2 P 6c8a58fdbe944622b27ade12cd92a62e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 02:52:28.278612 22345 raft_consensus.cc:3037] T 2a1eee69ea8549cbbb34c746f0fd69e2 P 6c8a58fdbe944622b27ade12cd92a62e [term 0 FOLLOWER]: Advancing to term 1
I20260811 02:52:28.279623 22345 raft_consensus.cc:504] T 2a1eee69ea8549cbbb34c746f0fd69e2 P 6c8a58fdbe944622b27ade12cd92a62e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6937bfa1e57344d78f8cc3b5df278e87" member_type: VOTER last_known_addr { host: "127.12.138.193" port: 34941 } } peers { permanent_uuid: "6c8a58fdbe944622b27ade12cd92a62e" member_type: VOTER last_known_addr { host: "127.12.138.194" port: 39121 } } peers { permanent_uuid: "6a04a1a9c41e4f7fb9064c78468ba498" member_type: VOTER last_known_addr { host: "127.12.138.195" port: 40285 } }
I20260811 02:52:28.279966 22345 leader_election.cc:288] T 2a1eee69ea8549cbbb34c746f0fd69e2 P 6c8a58fdbe944622b27ade12cd92a62e [CANDIDATE]: Term 1 election: Requested vote from peers 6937bfa1e57344d78f8cc3b5df278e87 (127.12.138.193:34941), 6a04a1a9c41e4f7fb9064c78468ba498 (127.12.138.195:40285)
I20260811 02:52:28.280272 21937 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2a1eee69ea8549cbbb34c746f0fd69e2" candidate_uuid: "6c8a58fdbe944622b27ade12cd92a62e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6937bfa1e57344d78f8cc3b5df278e87"
I20260811 02:52:28.280404 21937 raft_consensus.cc:3037] T 2a1eee69ea8549cbbb34c746f0fd69e2 P 6937bfa1e57344d78f8cc3b5df278e87 [term 0 FOLLOWER]: Advancing to term 1
I20260811 02:52:28.280563 22249 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2a1eee69ea8549cbbb34c746f0fd69e2" candidate_uuid: "6c8a58fdbe944622b27ade12cd92a62e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6a04a1a9c41e4f7fb9064c78468ba498"
I20260811 02:52:28.280689 22249 raft_consensus.cc:3037] T 2a1eee69ea8549cbbb34c746f0fd69e2 P 6a04a1a9c41e4f7fb9064c78468ba498 [term 0 FOLLOWER]: Advancing to term 1
I20260811 02:52:28.281780 22249 raft_consensus.cc:2454] T 2a1eee69ea8549cbbb34c746f0fd69e2 P 6a04a1a9c41e4f7fb9064c78468ba498 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c8a58fdbe944622b27ade12cd92a62e in term 1.
I20260811 02:52:28.281795 21937 raft_consensus.cc:2454] T 2a1eee69ea8549cbbb34c746f0fd69e2 P 6937bfa1e57344d78f8cc3b5df278e87 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c8a58fdbe944622b27ade12cd92a62e in term 1.
I20260811 02:52:28.282199 22016 leader_election.cc:302] T 2a1eee69ea8549cbbb34c746f0fd69e2 P 6c8a58fdbe944622b27ade12cd92a62e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a04a1a9c41e4f7fb9064c78468ba498, 6c8a58fdbe944622b27ade12cd92a62e; no voters:
I20260811 02:52:28.282403 22345 raft_consensus.cc:2781] T 2a1eee69ea8549cbbb34c746f0fd69e2 P 6c8a58fdbe944622b27ade12cd92a62e [term 1 FOLLOWER]: Leader election won for term 1
I20260811 02:52:28.282629 22345 raft_consensus.cc:686] T 2a1eee69ea8549cbbb34c746f0fd69e2 P 6c8a58fdbe944622b27ade12cd92a62e [term 1 LEADER]: Becoming Leader. State: Replica: 6c8a58fdbe944622b27ade12cd92a62e, State: Running, Role: LEADER
I20260811 02:52:28.282790 22345 consensus_queue.cc:237] T 2a1eee69ea8549cbbb34c746f0fd69e2 P 6c8a58fdbe944622b27ade12cd92a62e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6937bfa1e57344d78f8cc3b5df278e87" member_type: VOTER last_known_addr { host: "127.12.138.193" port: 34941 } } peers { permanent_uuid: "6c8a58fdbe944622b27ade12cd92a62e" member_type: VOTER last_known_addr { host: "127.12.138.194" port: 39121 } } peers { permanent_uuid: "6a04a1a9c41e4f7fb9064c78468ba498" member_type: VOTER last_known_addr { host: "127.12.138.195" port: 40285 } }
I20260811 02:52:28.284489 21759 catalog_manager.cc:5495] T 2a1eee69ea8549cbbb34c746f0fd69e2 P 6c8a58fdbe944622b27ade12cd92a62e reported cstate change: term changed from 0 to 1, leader changed from <none> to 6c8a58fdbe944622b27ade12cd92a62e (127.12.138.194). New cstate: current_term: 1 leader_uuid: "6c8a58fdbe944622b27ade12cd92a62e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6937bfa1e57344d78f8cc3b5df278e87" member_type: VOTER last_known_addr { host: "127.12.138.193" port: 34941 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6c8a58fdbe944622b27ade12cd92a62e" member_type: VOTER last_known_addr { host: "127.12.138.194" port: 39121 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6a04a1a9c41e4f7fb9064c78468ba498" member_type: VOTER last_known_addr { host: "127.12.138.195" port: 40285 } health_report { overall_health: UNKNOWN } } }
I20260811 02:52:28.301616 22431 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:52:28.298078 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40340 (local address 127.12.138.194:39121)
0811 02:52:28.298314 (+ 236us) server_negotiation.cc:206] Beginning negotiation
0811 02:52:28.298318 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 02:52:28.298330 (+ 12us) server_negotiation.cc:407] Connection header received
0811 02:52:28.298369 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:52:28.298370 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:52:28.298411 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:52:28.298485 (+ 74us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 02:52:28.298762 (+ 277us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:52:28.299295 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:52:28.300471 (+ 1176us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:52:28.300698 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:52:28.300785 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:52:28.301050 (+ 265us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 02:52:28.301120 (+ 70us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 02:52:28.301297 (+ 177us) server_negotiation.cc:1035] Waiting for connection context
0811 02:52:28.301401 (+ 104us) server_negotiation.cc:299] Negotiation successful
0811 02:52:28.301447 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":171,"thread_start_us":82,"threads_started":1}
I20260811 02:52:28.311775 22436 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:52:28.308172 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.138.193:34941 (local address 127.12.138.194:58409)
0811 02:52:28.308485 (+ 313us) negotiation.cc:107] Waiting for socket to connect
0811 02:52:28.308498 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 02:52:28.308585 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:52:28.308839 (+ 254us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:52:28.308842 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:52:28.308848 (+ 6us) client_negotiation.cc:189] Negotiated authn=TOKEN
0811 02:52:28.308929 (+ 81us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:52:28.308934 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:52:28.309943 (+ 1009us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:52:28.309946 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:52:28.310554 (+ 608us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:52:28.310564 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:52:28.311140 (+ 576us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:52:28.311142 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:52:28.311206 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:52:28.311247 (+ 41us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0811 02:52:28.311540 (+ 293us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0811 02:52:28.311545 (+ 5us) client_negotiation.cc:769] Sending connection context
0811 02:52:28.311594 (+ 49us) client_negotiation.cc:240] Negotiation successful
0811 02:52:28.311664 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":215,"thread_start_us":121,"threads_started":1}
I20260811 02:52:28.311789 22389 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:52:28.308305 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.138.194:58409 (local address 127.12.138.193:34941)
0811 02:52:28.308507 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0811 02:52:28.308512 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 02:52:28.308550 (+ 38us) server_negotiation.cc:407] Connection header received
0811 02:52:28.308685 (+ 135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:52:28.308688 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:52:28.308739 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:52:28.308819 (+ 80us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 02:52:28.309029 (+ 210us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:52:28.309832 (+ 803us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:52:28.310665 (+ 833us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:52:28.311030 (+ 365us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:52:28.311129 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:52:28.311378 (+ 249us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 02:52:28.311457 (+ 79us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 02:52:28.311525 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0811 02:52:28.311603 (+ 78us) server_negotiation.cc:299] Negotiation successful
0811 02:52:28.311658 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":89}
I20260811 02:52:28.317718 22249 raft_consensus.cc:1261] T 2a1eee69ea8549cbbb34c746f0fd69e2 P 6a04a1a9c41e4f7fb9064c78468ba498 [term 1 FOLLOWER]: Refusing update from remote peer 6c8a58fdbe944622b27ade12cd92a62e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 02:52:28.317741 21937 raft_consensus.cc:1261] T 2a1eee69ea8549cbbb34c746f0fd69e2 P 6937bfa1e57344d78f8cc3b5df278e87 [term 1 FOLLOWER]: Refusing update from remote peer 6c8a58fdbe944622b27ade12cd92a62e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 02:52:28.318136 22345 consensus_queue.cc:1035] T 2a1eee69ea8549cbbb34c746f0fd69e2 P 6c8a58fdbe944622b27ade12cd92a62e [LEADER]: Connected to new peer: Peer: permanent_uuid: "6937bfa1e57344d78f8cc3b5df278e87" member_type: VOTER last_known_addr { host: "127.12.138.193" port: 34941 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 02:52:28.318292 22429 consensus_queue.cc:1035] T 2a1eee69ea8549cbbb34c746f0fd69e2 P 6c8a58fdbe944622b27ade12cd92a62e [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a04a1a9c41e4f7fb9064c78468ba498" member_type: VOTER last_known_addr { host: "127.12.138.195" port: 40285 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 02:52:28.335480 22394 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:52:28.331848 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.138.194:39121 (local address 127.12.138.193:44743)
0811 02:52:28.332079 (+ 231us) negotiation.cc:107] Waiting for socket to connect
0811 02:52:28.332097 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 02:52:28.332211 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:52:28.332626 (+ 415us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:52:28.332629 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:52:28.332637 (+ 8us) client_negotiation.cc:189] Negotiated authn=TOKEN
0811 02:52:28.332746 (+ 109us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:52:28.332752 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:52:28.333638 (+ 886us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:52:28.333641 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:52:28.334233 (+ 592us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:52:28.334241 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:52:28.334719 (+ 478us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:52:28.334723 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:52:28.334827 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:52:28.334868 (+ 41us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0811 02:52:28.335178 (+ 310us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0811 02:52:28.335187 (+ 9us) client_negotiation.cc:769] Sending connection context
0811 02:52:28.335253 (+ 66us) client_negotiation.cc:240] Negotiation successful
0811 02:52:28.335360 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":137}
I20260811 02:52:28.335480 22431 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:52:28.332157 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.138.193:44743 (local address 127.12.138.194:39121)
0811 02:52:28.332277 (+ 120us) server_negotiation.cc:206] Beginning negotiation
0811 02:52:28.332281 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 02:52:28.332293 (+ 12us) server_negotiation.cc:407] Connection header received
0811 02:52:28.332332 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:52:28.332334 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:52:28.332403 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:52:28.332477 (+ 74us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 02:52:28.332898 (+ 421us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:52:28.333533 (+ 635us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:52:28.334336 (+ 803us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:52:28.334603 (+ 267us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:52:28.334701 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:52:28.335006 (+ 305us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 02:52:28.335088 (+ 82us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 02:52:28.335170 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0811 02:52:28.335260 (+ 90us) server_negotiation.cc:299] Negotiation successful
0811 02:52:28.335333 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260811 02:52:28.355175 21759 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36710:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 02:52:28.355355 21759 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36710:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 02:52:28.357548 21759 catalog_manager.cc:5782] T 00000000000000000000000000000000 P acf0fa3758a14749a4a0ec3e106b3dd5: Sending DeleteTablet for 3 replicas of tablet 2a1eee69ea8549cbbb34c746f0fd69e2
I20260811 02:52:28.358047 21915 tablet_service.cc:1514] Processing DeleteTablet for tablet 2a1eee69ea8549cbbb34c746f0fd69e2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 02:52:28 UTC) from {username='oryx', principal='oryx/127.12.138.254@KRBTEST.COM'} at 127.0.0.1:35364
I20260811 02:52:28.358177 22073 tablet_service.cc:1514] Processing DeleteTablet for tablet 2a1eee69ea8549cbbb34c746f0fd69e2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 02:52:28 UTC) from {username='oryx', principal='oryx/127.12.138.254@KRBTEST.COM'} at 127.0.0.1:40316
I20260811 02:52:28.358245 22401 tablet_replica.cc:330] T 2a1eee69ea8549cbbb34c746f0fd69e2 P 6937bfa1e57344d78f8cc3b5df278e87: stopping tablet replica
I20260811 02:52:28.358274 22225 tablet_service.cc:1514] Processing DeleteTablet for tablet 2a1eee69ea8549cbbb34c746f0fd69e2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 02:52:28 UTC) from {username='oryx', principal='oryx/127.12.138.254@KRBTEST.COM'} at 127.0.0.1:60306
I20260811 02:52:28.358363 22399 tablet_replica.cc:330] T 2a1eee69ea8549cbbb34c746f0fd69e2 P 6c8a58fdbe944622b27ade12cd92a62e: stopping tablet replica
I20260811 02:52:28.358402 22401 raft_consensus.cc:2229] T 2a1eee69ea8549cbbb34c746f0fd69e2 P 6937bfa1e57344d78f8cc3b5df278e87 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 02:52:28.358441 22400 tablet_replica.cc:330] T 2a1eee69ea8549cbbb34c746f0fd69e2 P 6a04a1a9c41e4f7fb9064c78468ba498: stopping tablet replica
I20260811 02:52:28.358496 22399 raft_consensus.cc:2229] T 2a1eee69ea8549cbbb34c746f0fd69e2 P 6c8a58fdbe944622b27ade12cd92a62e [term 1 LEADER]: Raft consensus shutting down.
I20260811 02:52:28.358527 22401 raft_consensus.cc:2258] T 2a1eee69ea8549cbbb34c746f0fd69e2 P 6937bfa1e57344d78f8cc3b5df278e87 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 02:52:28.358546 22400 raft_consensus.cc:2229] T 2a1eee69ea8549cbbb34c746f0fd69e2 P 6a04a1a9c41e4f7fb9064c78468ba498 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 02:52:28.358644 22400 raft_consensus.cc:2258] T 2a1eee69ea8549cbbb34c746f0fd69e2 P 6a04a1a9c41e4f7fb9064c78468ba498 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 02:52:28.358774 22399 raft_consensus.cc:2258] T 2a1eee69ea8549cbbb34c746f0fd69e2 P 6c8a58fdbe944622b27ade12cd92a62e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 02:52:28.359231 22401 ts_tablet_manager.cc:1905] T 2a1eee69ea8549cbbb34c746f0fd69e2 P 6937bfa1e57344d78f8cc3b5df278e87: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 02:52:28.359409 22400 ts_tablet_manager.cc:1905] T 2a1eee69ea8549cbbb34c746f0fd69e2 P 6a04a1a9c41e4f7fb9064c78468ba498: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 02:52:28.359436 22399 ts_tablet_manager.cc:1905] T 2a1eee69ea8549cbbb34c746f0fd69e2 P 6c8a58fdbe944622b27ade12cd92a62e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 02:52:28.361625 22401 ts_tablet_manager.cc:1918] T 2a1eee69ea8549cbbb34c746f0fd69e2 P 6937bfa1e57344d78f8cc3b5df278e87: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 02:52:28.361721 22401 log.cc:1196] T 2a1eee69ea8549cbbb34c746f0fd69e2 P 6937bfa1e57344d78f8cc3b5df278e87: Deleting WAL directory at /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786416707705494-12843-0/minicluster-data/ts-0/wal/wals/2a1eee69ea8549cbbb34c746f0fd69e2
I20260811 02:52:28.361879 22399 ts_tablet_manager.cc:1918] T 2a1eee69ea8549cbbb34c746f0fd69e2 P 6c8a58fdbe944622b27ade12cd92a62e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 02:52:28.361879 22400 ts_tablet_manager.cc:1918] T 2a1eee69ea8549cbbb34c746f0fd69e2 P 6a04a1a9c41e4f7fb9064c78468ba498: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 02:52:28.361963 22400 log.cc:1196] T 2a1eee69ea8549cbbb34c746f0fd69e2 P 6a04a1a9c41e4f7fb9064c78468ba498: Deleting WAL directory at /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786416707705494-12843-0/minicluster-data/ts-2/wal/wals/2a1eee69ea8549cbbb34c746f0fd69e2
I20260811 02:52:28.361963 22399 log.cc:1196] T 2a1eee69ea8549cbbb34c746f0fd69e2 P 6c8a58fdbe944622b27ade12cd92a62e: Deleting WAL directory at /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786416707705494-12843-0/minicluster-data/ts-1/wal/wals/2a1eee69ea8549cbbb34c746f0fd69e2
I20260811 02:52:28.362043 22401 ts_tablet_manager.cc:1939] T 2a1eee69ea8549cbbb34c746f0fd69e2 P 6937bfa1e57344d78f8cc3b5df278e87: Deleting consensus metadata
I20260811 02:52:28.362253 22399 ts_tablet_manager.cc:1939] T 2a1eee69ea8549cbbb34c746f0fd69e2 P 6c8a58fdbe944622b27ade12cd92a62e: Deleting consensus metadata
I20260811 02:52:28.362253 22400 ts_tablet_manager.cc:1939] T 2a1eee69ea8549cbbb34c746f0fd69e2 P 6a04a1a9c41e4f7fb9064c78468ba498: Deleting consensus metadata
I20260811 02:52:28.362730 21745 catalog_manager.cc:4841] TS 6937bfa1e57344d78f8cc3b5df278e87 (127.12.138.193:34941): tablet 2a1eee69ea8549cbbb34c746f0fd69e2 (table test-table [id=8ee467ecab784d0a93757a92c0929fd7]) successfully deleted
I20260811 02:52:28.362869 21744 catalog_manager.cc:4841] TS 6a04a1a9c41e4f7fb9064c78468ba498 (127.12.138.195:40285): tablet 2a1eee69ea8549cbbb34c746f0fd69e2 (table test-table [id=8ee467ecab784d0a93757a92c0929fd7]) successfully deleted
I20260811 02:52:28.363058 21746 catalog_manager.cc:4841] TS 6c8a58fdbe944622b27ade12cd92a62e (127.12.138.194:39121): tablet 2a1eee69ea8549cbbb34c746f0fd69e2 (table test-table [id=8ee467ecab784d0a93757a92c0929fd7]) successfully deleted
I20260811 02:52:28.363587 12843 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDawEwS/build/debug/bin/kudu with pid 21841
I20260811 02:52:28.376156 12843 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDawEwS/build/debug/bin/kudu with pid 21999
I20260811 02:52:28.386458 12843 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDawEwS/build/debug/bin/kudu with pid 22155
I20260811 02:52:28.395442 12843 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDawEwS/build/debug/bin/kudu with pid 21725
2026-08-11T02:52:28Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipal (3572 ms)
[ RUN ] SecurityITest.TestNonExistentPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786416707705494-12843-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 02:52:28 dist-test-slave-9gcw krb5kdc[22442](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 02:52:28 dist-test-slave-9gcw krb5kdc[22442](info): set up 2 sockets
Aug 11 02:52:28 dist-test-slave-9gcw krb5kdc[22442](info): commencing operation
krb5kdc: starting...
W20260811 02:52:29.551810 12843 mini_kdc.cc:121] Time spent starting KDC: real 1.131s user 0.002s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786416707705494-12843-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786416707705494-12843-0/krb5kdc/test-user.keytab.
Aug 11 02:52:29 dist-test-slave-9gcw krb5kdc[22442](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416749, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T02:52:29Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T02:52:29Z Disabled control of system clock
WARNING: no policy specified for kudu/127.12.138.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.138.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.138.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786416707705494-12843-0/krb5kdc/kudu_127.12.138.254.keytab.
Entry for principal kudu/127.12.138.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786416707705494-12843-0/krb5kdc/kudu_127.12.138.254.keytab.
I20260811 02:52:29.692293 12843 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDawEwS/build/debug/bin/kudu
/tmp/dist-test-taskDawEwS/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786416707705494-12843-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786416707705494-12843-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786416707705494-12843-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786416707705494-12843-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.138.254:34139
--webserver_interface=127.12.138.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786416707705494-12843-0/krb5kdc/kudu_127.12.138.254.keytab
--principal=kudu/127.12.138.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.138.212:40997
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.12.138.254:34139
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--principal=oryx with env {KRB5CCNAME=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786416707705494-12843-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786416707705494-12843-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786416707705494-12843-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 02:52:29.808326 22457 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 02:52:29.808617 22457 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 02:52:29.808657 22457 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 02:52:29.812820 22457 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 02:52:29.812898 22457 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 02:52:29.812917 22457 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 02:52:29.812963 22457 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 02:52:29.813001 22457 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 02:52:29.818519 22457 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.138.212:40997
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786416707705494-12843-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786416707705494-12843-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.12.138.254:34139
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786416707705494-12843-0/krb5kdc/kudu_127.12.138.254.keytab
--principal=oryx
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.138.254:34139
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786416707705494-12843-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.12.138.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22457
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786416707705494-12843-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 02:09:49 UTC on ccycloud.quasar-fsnqqc.root.comops.site
build id 4579570
I20260811 02:52:29.819659 22457 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 02:52:29.820719 22457 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 02:52:29.829344 22468 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:52:29.829352 22466 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:52:29.829352 22465 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:52:29.829612 22457 server_base.cc:1034] running on GCE node
I20260811 02:52:29.830281 22457 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 02:52:29.831476 22457 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 02:52:29.832665 22457 hybrid_clock.cc:648] HybridClock initialized: now 1786416749832651 us; error 29 us; skew 500 ppm
Runtime error: kinit failed: unable to login from keytab: Keytab contains no suitable keys for oryx@KRBTEST.COM
2026-08-11T02:52:29Z chronyd exiting
[ OK ] SecurityITest.TestNonExistentPrincipal (1452 ms)
[ RUN ] SecurityITest.TestMismatchingPrincipals
Loading random data
Initializing database '/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786416707705494-12843-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 02:52:29 dist-test-slave-9gcw krb5kdc[22473](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 02:52:29 dist-test-slave-9gcw krb5kdc[22473](info): set up 2 sockets
Aug 11 02:52:29 dist-test-slave-9gcw krb5kdc[22473](info): commencing operation
krb5kdc: starting...
W20260811 02:52:31.100805 12843 mini_kdc.cc:121] Time spent starting KDC: real 1.227s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786416707705494-12843-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786416707705494-12843-0/krb5kdc/test-user.keytab.
Aug 11 02:52:31 dist-test-slave-9gcw krb5kdc[22473](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416751, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T02:52:31Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T02:52:31Z Disabled control of system clock
WARNING: no policy specified for kudu/127.12.138.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.138.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.138.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786416707705494-12843-0/krb5kdc/kudu_127.12.138.254.keytab.
Entry for principal kudu/127.12.138.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786416707705494-12843-0/krb5kdc/kudu_127.12.138.254.keytab.
I20260811 02:52:31.234391 12843 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDawEwS/build/debug/bin/kudu
/tmp/dist-test-taskDawEwS/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786416707705494-12843-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786416707705494-12843-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786416707705494-12843-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786416707705494-12843-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.138.254:34683
--webserver_interface=127.12.138.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786416707705494-12843-0/krb5kdc/kudu_127.12.138.254.keytab
--principal=kudu/127.12.138.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.138.212:37315
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.12.138.254:34683
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786416707705494-12843-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786416707705494-12843-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786416707705494-12843-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 02:52:31.350713 22486 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 02:52:31.351037 22486 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 02:52:31.351135 22486 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 02:52:31.355209 22486 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 02:52:31.355306 22486 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 02:52:31.355365 22486 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 02:52:31.355424 22486 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 02:52:31.355464 22486 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 02:52:31.360745 22486 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.138.212:37315
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786416707705494-12843-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786416707705494-12843-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.12.138.254:34683
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786416707705494-12843-0/krb5kdc/kudu_127.12.138.254.keytab
--principal=kudu/127.12.138.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.138.254:34683
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786416707705494-12843-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.12.138.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22486
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786416707705494-12843-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 02:09:49 UTC on ccycloud.quasar-fsnqqc.root.comops.site
build id 4579570
I20260811 02:52:31.361871 22486 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 02:52:31.362914 22486 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 02:52:31.371440 22494 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:52:31.371518 22486 server_base.cc:1034] running on GCE node
W20260811 02:52:31.371475 22491 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:52:31.371474 22492 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:52:31.372289 22486 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 02:52:31.373482 22486 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 02:52:31.374673 22486 hybrid_clock.cc:648] HybridClock initialized: now 1786416751374659 us; error 29 us; skew 500 ppm
Aug 11 02:52:31 dist-test-slave-9gcw krb5kdc[22473](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416751, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.138.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 02:52:31.377516 22486 init.cc:377] Logged in from keytab as kudu/127.12.138.254@KRBTEST.COM (short username kudu)
I20260811 02:52:31.378659 22486 webserver.cc:460] Webserver started at http://127.12.138.254:36273/ using document root <none> and password file <none>
I20260811 02:52:31.379347 22486 fs_manager.cc:359] Metadata directory not provided
I20260811 02:52:31.379455 22486 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 02:52:31.379741 22486 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 02:52:31.382354 22486 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786416707705494-12843-0/minicluster-data/master-0/data/instance:
uuid: "b18a0d21a30e49938b995a4676caaad8"
format_stamp: "Formatted at 2026-08-11 02:52:31 on dist-test-slave-9gcw"
server_key: "c7546f4b32447ef65eba18e28fe36f26"
server_key_iv: "1909c7f5778486e0303a15fc296e0df0"
server_key_version: "encryptionkey@0"
I20260811 02:52:31.383144 22486 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786416707705494-12843-0/minicluster-data/master-0/wal/instance:
uuid: "b18a0d21a30e49938b995a4676caaad8"
format_stamp: "Formatted at 2026-08-11 02:52:31 on dist-test-slave-9gcw"
server_key: "c7546f4b32447ef65eba18e28fe36f26"
server_key_iv: "1909c7f5778486e0303a15fc296e0df0"
server_key_version: "encryptionkey@0"
I20260811 02:52:31.387794 22486 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260811 02:52:31.390560 22501 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 02:52:31.391893 22486 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260811 02:52:31.392072 22486 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786416707705494-12843-0/minicluster-data/master-0/data,/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786416707705494-12843-0/minicluster-data/master-0/wal
uuid: "b18a0d21a30e49938b995a4676caaad8"
format_stamp: "Formatted at 2026-08-11 02:52:31 on dist-test-slave-9gcw"
server_key: "c7546f4b32447ef65eba18e28fe36f26"
server_key_iv: "1909c7f5778486e0303a15fc296e0df0"
server_key_version: "encryptionkey@0"
I20260811 02:52:31.392206 22486 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786416707705494-12843-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786416707705494-12843-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786416707705494-12843-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 02:52:31.404001 22486 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 02:52:31.406989 22486 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 02:52:31.407224 22486 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 02:52:31.414898 22486 rpc_server.cc:307] RPC server started. Bound to: 127.12.138.254:34683
I20260811 02:52:31.414937 22558 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.138.254:34683 every 8 connection(s)
I20260811 02:52:31.415922 22486 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786416707705494-12843-0/minicluster-data/master-0/data/info.pb
I20260811 02:52:31.418774 22560 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 02:52:31.421799 12843 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDawEwS/build/debug/bin/kudu as pid 22486
I20260811 02:52:31.421955 12843 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786416707705494-12843-0/minicluster-data/master-0/wal/instance
I20260811 02:52:31.422292 12843 external_mini_cluster.cc:1428] Setting key ed7e4561186e54dc749032c8a5c9450c
I20260811 02:52:31.425597 22560 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b18a0d21a30e49938b995a4676caaad8: Bootstrap starting.
I20260811 02:52:31.427915 22560 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b18a0d21a30e49938b995a4676caaad8: Neither blocks nor log segments found. Creating new log.
I20260811 02:52:31.428751 22560 log.cc:824] T 00000000000000000000000000000000 P b18a0d21a30e49938b995a4676caaad8: Log is configured to *not* fsync() on all Append() calls
I20260811 02:52:31.430823 22560 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b18a0d21a30e49938b995a4676caaad8: No bootstrap required, opened a new log
Aug 11 02:52:31 dist-test-slave-9gcw krb5kdc[22473](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416751, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.12.138.254@KRBTEST.COM
I20260811 02:52:31.434394 22560 raft_consensus.cc:348] T 00000000000000000000000000000000 P b18a0d21a30e49938b995a4676caaad8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b18a0d21a30e49938b995a4676caaad8" member_type: VOTER last_known_addr { host: "127.12.138.254" port: 34683 } }
I20260811 02:52:31.434651 22560 raft_consensus.cc:374] T 00000000000000000000000000000000 P b18a0d21a30e49938b995a4676caaad8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 02:52:31.434708 22560 raft_consensus.cc:729] T 00000000000000000000000000000000 P b18a0d21a30e49938b995a4676caaad8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b18a0d21a30e49938b995a4676caaad8, State: Initialized, Role: FOLLOWER
I20260811 02:52:31.435207 22560 consensus_queue.cc:260] T 00000000000000000000000000000000 P b18a0d21a30e49938b995a4676caaad8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b18a0d21a30e49938b995a4676caaad8" member_type: VOTER last_known_addr { host: "127.12.138.254" port: 34683 } }
I20260811 02:52:31.435406 22560 raft_consensus.cc:388] T 00000000000000000000000000000000 P b18a0d21a30e49938b995a4676caaad8 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 02:52:31.435490 22560 raft_consensus.cc:482] T 00000000000000000000000000000000 P b18a0d21a30e49938b995a4676caaad8 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 02:52:31.435613 22560 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b18a0d21a30e49938b995a4676caaad8 [term 0 FOLLOWER]: Advancing to term 1
I20260811 02:52:31.436631 22560 raft_consensus.cc:504] T 00000000000000000000000000000000 P b18a0d21a30e49938b995a4676caaad8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b18a0d21a30e49938b995a4676caaad8" member_type: VOTER last_known_addr { host: "127.12.138.254" port: 34683 } }
I20260811 02:52:31.437002 22560 leader_election.cc:302] T 00000000000000000000000000000000 P b18a0d21a30e49938b995a4676caaad8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b18a0d21a30e49938b995a4676caaad8; no voters:
I20260811 02:52:31.437409 22560 leader_election.cc:288] T 00000000000000000000000000000000 P b18a0d21a30e49938b995a4676caaad8 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 02:52:31.437493 22565 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b18a0d21a30e49938b995a4676caaad8 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 02:52:31.437729 22565 raft_consensus.cc:686] T 00000000000000000000000000000000 P b18a0d21a30e49938b995a4676caaad8 [term 1 LEADER]: Becoming Leader. State: Replica: b18a0d21a30e49938b995a4676caaad8, State: Running, Role: LEADER
I20260811 02:52:31.437767 22563 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:52:31.423913 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57820 (local address 127.12.138.254:34683)
0811 02:52:31.424403 (+ 490us) server_negotiation.cc:206] Beginning negotiation
0811 02:52:31.424419 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0811 02:52:31.424457 (+ 38us) server_negotiation.cc:407] Connection header received
0811 02:52:31.425125 (+ 668us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:52:31.425223 (+ 98us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:52:31.425579 (+ 356us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:52:31.425982 (+ 403us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:52:31.426854 (+ 872us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:52:31.427896 (+ 1042us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:52:31.428841 (+ 945us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:52:31.429465 (+ 624us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:52:31.429657 (+ 192us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:52:31.432344 (+ 2687us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:52:31.432349 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:52:31.432363 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:52:31.432391 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:52:31.434248 (+ 1857us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:52:31.434740 (+ 492us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:52:31.434749 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:52:31.434755 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:52:31.434909 (+ 154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:52:31.435218 (+ 309us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:52:31.435222 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:52:31.435224 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:52:31.435842 (+ 618us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:52:31.436057 (+ 215us) server_negotiation.cc:1035] Waiting for connection context
0811 02:52:31.436382 (+ 325us) server_negotiation.cc:299] Negotiation successful
0811 02:52:31.436732 (+ 350us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":2,"server-negotiator.queue_time_us":340,"thread_start_us":150,"threads_started":1}
I20260811 02:52:31.438287 22565 consensus_queue.cc:237] T 00000000000000000000000000000000 P b18a0d21a30e49938b995a4676caaad8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b18a0d21a30e49938b995a4676caaad8" member_type: VOTER last_known_addr { host: "127.12.138.254" port: 34683 } }
I20260811 02:52:31.438809 22560 sys_catalog.cc:564] T 00000000000000000000000000000000 P b18a0d21a30e49938b995a4676caaad8 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 02:52:31.440052 22566 sys_catalog.cc:455] T 00000000000000000000000000000000 P b18a0d21a30e49938b995a4676caaad8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b18a0d21a30e49938b995a4676caaad8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b18a0d21a30e49938b995a4676caaad8" member_type: VOTER last_known_addr { host: "127.12.138.254" port: 34683 } } }
I20260811 02:52:31.440191 22566 sys_catalog.cc:458] T 00000000000000000000000000000000 P b18a0d21a30e49938b995a4676caaad8 [sys.catalog]: This master's current role is: LEADER
I20260811 02:52:31.440418 22566 sys_catalog.cc:455] T 00000000000000000000000000000000 P b18a0d21a30e49938b995a4676caaad8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b18a0d21a30e49938b995a4676caaad8. Latest consensus state: current_term: 1 leader_uuid: "b18a0d21a30e49938b995a4676caaad8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b18a0d21a30e49938b995a4676caaad8" member_type: VOTER last_known_addr { host: "127.12.138.254" port: 34683 } } }
I20260811 02:52:31.440505 22566 sys_catalog.cc:458] T 00000000000000000000000000000000 P b18a0d21a30e49938b995a4676caaad8 [sys.catalog]: This master's current role is: LEADER
I20260811 02:52:31.441583 22575 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 02:52:31.444389 22575 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 02:52:31.450278 22575 catalog_manager.cc:1347] Generated new cluster ID: dfaaf1425509400a98919b096d81a016
I20260811 02:52:31.450366 22575 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 02:52:31.456895 22575 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 02:52:31.458330 22575 catalog_manager.cc:1504] Loading token signing keys...
I20260811 02:52:31.466790 22575 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b18a0d21a30e49938b995a4676caaad8: Generated new TSK 0
I20260811 02:52:31.467775 22575 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.12.138.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.138.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.138.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786416707705494-12843-0/krb5kdc/kudu_127.12.138.193.keytab.
Entry for principal kudu/127.12.138.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786416707705494-12843-0/krb5kdc/kudu_127.12.138.193.keytab.
I20260811 02:52:31.500185 12843 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDawEwS/build/debug/bin/kudu
/tmp/dist-test-taskDawEwS/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786416707705494-12843-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786416707705494-12843-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786416707705494-12843-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786416707705494-12843-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.138.193:0
--local_ip_for_outbound_sockets=127.12.138.193
--webserver_interface=127.12.138.193
--webserver_port=0
--tserver_master_addrs=127.12.138.254:34683
--keytab_file=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786416707705494-12843-0/krb5kdc/kudu_127.12.138.193.keytab
--principal=kudu/127.12.138.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.138.212:37315
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786416707705494-12843-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786416707705494-12843-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786416707705494-12843-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 02:52:31.619508 22588 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 02:52:31.619894 22588 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 02:52:31.619941 22588 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 02:52:31.624291 22588 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 02:52:31.624368 22588 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 02:52:31.624430 22588 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.138.193
I20260811 02:52:31.629920 22588 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.138.212:37315
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786416707705494-12843-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786416707705494-12843-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786416707705494-12843-0/krb5kdc/kudu_127.12.138.193.keytab
--principal=kudu/127.12.138.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.138.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786416707705494-12843-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.12.138.193
--webserver_port=0
--tserver_master_addrs=127.12.138.254:34683
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22588
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.138.193
--log_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786416707705494-12843-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 02:09:49 UTC on ccycloud.quasar-fsnqqc.root.comops.site
build id 4579570
I20260811 02:52:31.631093 22588 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 02:52:31.632124 22588 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 02:52:31.643136 22598 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:52:31.643118 22595 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:52:31.643121 22596 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:52:31.643213 22588 server_base.cc:1034] running on GCE node
I20260811 02:52:31.643745 22588 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 02:52:31.644405 22588 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 02:52:31.645618 22588 hybrid_clock.cc:648] HybridClock initialized: now 1786416751645606 us; error 37 us; skew 500 ppm
Aug 11 02:52:31 dist-test-slave-9gcw krb5kdc[22473](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416751, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.138.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 02:52:31.648543 22588 init.cc:377] Logged in from keytab as kudu/127.12.138.193@KRBTEST.COM (short username kudu)
I20260811 02:52:31.649624 22588 webserver.cc:460] Webserver started at http://127.12.138.193:41325/ using document root <none> and password file <none>
I20260811 02:52:31.650255 22588 fs_manager.cc:359] Metadata directory not provided
I20260811 02:52:31.650372 22588 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 02:52:31.650625 22588 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 02:52:31.652782 22588 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786416707705494-12843-0/minicluster-data/ts-0/data/instance:
uuid: "d3a1386780b941a6b10f550a18c16fb9"
format_stamp: "Formatted at 2026-08-11 02:52:31 on dist-test-slave-9gcw"
server_key: "e10f6a7896ef05f7dc9db876f8142907"
server_key_iv: "fabe4486b36618567fa9fe9a00e19bf6"
server_key_version: "encryptionkey@0"
I20260811 02:52:31.653426 22588 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786416707705494-12843-0/minicluster-data/ts-0/wal/instance:
uuid: "d3a1386780b941a6b10f550a18c16fb9"
format_stamp: "Formatted at 2026-08-11 02:52:31 on dist-test-slave-9gcw"
server_key: "e10f6a7896ef05f7dc9db876f8142907"
server_key_iv: "fabe4486b36618567fa9fe9a00e19bf6"
server_key_version: "encryptionkey@0"
I20260811 02:52:31.657335 22588 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260811 02:52:31.660050 22607 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 02:52:31.661427 22588 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 02:52:31.661569 22588 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786416707705494-12843-0/minicluster-data/ts-0/data,/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786416707705494-12843-0/minicluster-data/ts-0/wal
uuid: "d3a1386780b941a6b10f550a18c16fb9"
format_stamp: "Formatted at 2026-08-11 02:52:31 on dist-test-slave-9gcw"
server_key: "e10f6a7896ef05f7dc9db876f8142907"
server_key_iv: "fabe4486b36618567fa9fe9a00e19bf6"
server_key_version: "encryptionkey@0"
I20260811 02:52:31.661688 22588 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786416707705494-12843-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786416707705494-12843-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786416707705494-12843-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 02:52:31.678377 22588 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 02:52:31.681565 22588 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 02:52:31.681813 22588 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 02:52:31.682502 22588 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 02:52:31.683465 22588 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 02:52:31.683542 22588 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 02:52:31.683622 22588 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 02:52:31.683689 22588 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 02:52:31.693405 22588 rpc_server.cc:307] RPC server started. Bound to: 127.12.138.193:36935
I20260811 02:52:31.693434 22739 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.138.193:36935 every 8 connection(s)
I20260811 02:52:31.694481 22588 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786416707705494-12843-0/minicluster-data/ts-0/data/info.pb
I20260811 02:52:31.698197 12843 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDawEwS/build/debug/bin/kudu as pid 22588
I20260811 02:52:31.698347 12843 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786416707705494-12843-0/minicluster-data/ts-0/wal/instance
I20260811 02:52:31.698664 12843 external_mini_cluster.cc:1428] Setting key cb254052bcc52fddf6b7925cd23e032d
Aug 11 02:52:31 dist-test-slave-9gcw krb5kdc[22473](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416751, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.138.193@KRBTEST.COM for kudu/127.12.138.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.12.138.194@KRBTEST.COM; defaulting to no policy
I20260811 02:52:31.709718 22563 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:52:31.696469 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.138.193:42195 (local address 127.12.138.254:34683)
0811 02:52:31.696634 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0811 02:52:31.696638 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 02:52:31.697568 (+ 930us) server_negotiation.cc:407] Connection header received
0811 02:52:31.699076 (+ 1508us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:52:31.699080 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:52:31.699143 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:52:31.699255 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:52:31.700456 (+ 1201us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:52:31.701000 (+ 544us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:52:31.702366 (+ 1366us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:52:31.702598 (+ 232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:52:31.702688 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:52:31.705820 (+ 3132us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:52:31.705824 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:52:31.705826 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:52:31.705852 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:52:31.707402 (+ 1550us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:52:31.707977 (+ 575us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:52:31.707986 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:52:31.707987 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:52:31.708069 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:52:31.708443 (+ 374us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:52:31.708446 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:52:31.708448 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:52:31.708652 (+ 204us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:52:31.708768 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0811 02:52:31.709380 (+ 612us) server_negotiation.cc:299] Negotiation successful
0811 02:52:31.709528 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260811 02:52:31.710603 22742 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:52:31.696919 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.138.254:34683 (local address 127.12.138.193:42195)
0811 02:52:31.697410 (+ 491us) negotiation.cc:107] Waiting for socket to connect
0811 02:52:31.697454 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0811 02:52:31.698854 (+ 1400us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:52:31.699444 (+ 590us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:52:31.699451 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:52:31.699819 (+ 368us) client_negotiation.cc:189] Negotiated authn=SASL
0811 02:52:31.700306 (+ 487us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:52:31.700320 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:52:31.701134 (+ 814us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:52:31.701138 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:52:31.702216 (+ 1078us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:52:31.702231 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:52:31.702764 (+ 533us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:52:31.702771 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:52:31.703021 (+ 250us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:52:31.703688 (+ 667us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 02:52:31.703714 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0811 02:52:31.705629 (+ 1915us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 02:52:31.707541 (+ 1912us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:52:31.707547 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:52:31.707564 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:52:31.707855 (+ 291us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:52:31.708167 (+ 312us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:52:31.708172 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:52:31.708174 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:52:31.708329 (+ 155us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:52:31.708767 (+ 438us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 02:52:31.708784 (+ 17us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 02:52:31.709061 (+ 277us) client_negotiation.cc:769] Sending connection context
0811 02:52:31.709312 (+ 251us) client_negotiation.cc:240] Negotiation successful
0811 02:52:31.709673 (+ 361us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":300,"thread_start_us":155,"threads_started":1}
I20260811 02:52:31.711954 22740 heartbeater.cc:347] Connected to a master server at 127.12.138.254:34683
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.138.194@KRBTEST.COM" created.
I20260811 02:52:31.712257 22740 heartbeater.cc:464] Registering TS with master...
I20260811 02:52:31.712862 22740 heartbeater.cc:511] Master 127.12.138.254:34683 requested a full tablet report, sending...
I20260811 02:52:31.714763 22519 ts_manager.cc:194] Registered new tserver with Master: d3a1386780b941a6b10f550a18c16fb9 (127.12.138.193:36935)
I20260811 02:52:31.715971 22519 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.12.138.193@KRBTEST.COM'} at 127.12.138.193:42195
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.138.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786416707705494-12843-0/krb5kdc/kudu_127.12.138.194.keytab.
Entry for principal kudu/127.12.138.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786416707705494-12843-0/krb5kdc/kudu_127.12.138.194.keytab.
I20260811 02:52:31.729709 12843 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDawEwS/build/debug/bin/kudu
/tmp/dist-test-taskDawEwS/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786416707705494-12843-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786416707705494-12843-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786416707705494-12843-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786416707705494-12843-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.138.194:0
--local_ip_for_outbound_sockets=127.12.138.194
--webserver_interface=127.12.138.194
--webserver_port=0
--tserver_master_addrs=127.12.138.254:34683
--keytab_file=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786416707705494-12843-0/krb5kdc/kudu_127.12.138.194.keytab
--principal=kudu/127.12.138.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.138.212:37315
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786416707705494-12843-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786416707705494-12843-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786416707705494-12843-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 02:52:31.848971 22746 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 02:52:31.849426 22746 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 02:52:31.849483 22746 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 02:52:31.853874 22746 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 02:52:31.853947 22746 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 02:52:31.854009 22746 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.138.194
I20260811 02:52:31.859674 22746 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.138.212:37315
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786416707705494-12843-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786416707705494-12843-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786416707705494-12843-0/krb5kdc/kudu_127.12.138.194.keytab
--principal=kudu/127.12.138.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.138.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786416707705494-12843-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.12.138.194
--webserver_port=0
--tserver_master_addrs=127.12.138.254:34683
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22746
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.138.194
--log_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786416707705494-12843-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 02:09:49 UTC on ccycloud.quasar-fsnqqc.root.comops.site
build id 4579570
I20260811 02:52:31.860836 22746 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 02:52:31.861940 22746 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 02:52:31.872409 22751 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:52:31.872400 22752 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:52:31.872552 22746 server_base.cc:1034] running on GCE node
W20260811 02:52:31.872400 22754 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:52:31.873263 22746 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 02:52:31.874022 22746 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 02:52:31.875214 22746 hybrid_clock.cc:648] HybridClock initialized: now 1786416751875190 us; error 43 us; skew 500 ppm
Aug 11 02:52:31 dist-test-slave-9gcw krb5kdc[22473](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416751, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.138.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 02:52:31.878321 22746 init.cc:377] Logged in from keytab as kudu/127.12.138.194@KRBTEST.COM (short username kudu)
I20260811 02:52:31.879264 22746 webserver.cc:460] Webserver started at http://127.12.138.194:37063/ using document root <none> and password file <none>
I20260811 02:52:31.879868 22746 fs_manager.cc:359] Metadata directory not provided
I20260811 02:52:31.879966 22746 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 02:52:31.880210 22746 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 02:52:31.882373 22746 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786416707705494-12843-0/minicluster-data/ts-1/data/instance:
uuid: "737681acde44467ca0932902a31fabc8"
format_stamp: "Formatted at 2026-08-11 02:52:31 on dist-test-slave-9gcw"
server_key: "844e47d595cab1b56681859b29f1576f"
server_key_iv: "c6d1187d3b7433dd393b38f4b23e1646"
server_key_version: "encryptionkey@0"
I20260811 02:52:31.882972 22746 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786416707705494-12843-0/minicluster-data/ts-1/wal/instance:
uuid: "737681acde44467ca0932902a31fabc8"
format_stamp: "Formatted at 2026-08-11 02:52:31 on dist-test-slave-9gcw"
server_key: "844e47d595cab1b56681859b29f1576f"
server_key_iv: "c6d1187d3b7433dd393b38f4b23e1646"
server_key_version: "encryptionkey@0"
I20260811 02:52:31.886971 22746 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260811 02:52:31.889562 22761 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 02:52:31.890864 22746 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260811 02:52:31.891004 22746 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786416707705494-12843-0/minicluster-data/ts-1/data,/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786416707705494-12843-0/minicluster-data/ts-1/wal
uuid: "737681acde44467ca0932902a31fabc8"
format_stamp: "Formatted at 2026-08-11 02:52:31 on dist-test-slave-9gcw"
server_key: "844e47d595cab1b56681859b29f1576f"
server_key_iv: "c6d1187d3b7433dd393b38f4b23e1646"
server_key_version: "encryptionkey@0"
I20260811 02:52:31.891137 22746 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786416707705494-12843-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786416707705494-12843-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786416707705494-12843-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 02:52:31.906952 22746 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 02:52:31.910626 22746 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 02:52:31.910859 22746 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 02:52:31.911480 22746 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 02:52:31.912344 22746 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 02:52:31.912422 22746 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 02:52:31.912495 22746 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 02:52:31.912542 22746 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 02:52:31.922560 22746 rpc_server.cc:307] RPC server started. Bound to: 127.12.138.194:41475
I20260811 02:52:31.922582 22892 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.138.194:41475 every 8 connection(s)
I20260811 02:52:31.923554 22746 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786416707705494-12843-0/minicluster-data/ts-1/data/info.pb
I20260811 02:52:31.927922 12843 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDawEwS/build/debug/bin/kudu as pid 22746
I20260811 02:52:31.928094 12843 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786416707705494-12843-0/minicluster-data/ts-1/wal/instance
I20260811 02:52:31.928423 12843 external_mini_cluster.cc:1428] Setting key ae646dffbfe09b9f4cabafb103db7d45
Aug 11 02:52:31 dist-test-slave-9gcw krb5kdc[22473](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416751, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.138.194@KRBTEST.COM for kudu/127.12.138.254@KRBTEST.COM
I20260811 02:52:31.937785 22563 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:52:31.925388 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.138.194:46921 (local address 127.12.138.254:34683)
0811 02:52:31.925558 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0811 02:52:31.925563 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 02:52:31.926452 (+ 889us) server_negotiation.cc:407] Connection header received
0811 02:52:31.927712 (+ 1260us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:52:31.927716 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:52:31.927779 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:52:31.927884 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:52:31.929394 (+ 1510us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:52:31.930044 (+ 650us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:52:31.931181 (+ 1137us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:52:31.931423 (+ 242us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:52:31.931528 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:52:31.934084 (+ 2556us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:52:31.934087 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:52:31.934090 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:52:31.934115 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:52:31.935603 (+ 1488us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:52:31.936130 (+ 527us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:52:31.936133 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:52:31.936135 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:52:31.936217 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:52:31.936560 (+ 343us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:52:31.936563 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:52:31.936565 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:52:31.936771 (+ 206us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:52:31.936879 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0811 02:52:31.937482 (+ 603us) server_negotiation.cc:299] Negotiation successful
0811 02:52:31.937610 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260811 02:52:31.938623 22895 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:52:31.925813 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.138.254:34683 (local address 127.12.138.194:46921)
0811 02:52:31.926269 (+ 456us) negotiation.cc:107] Waiting for socket to connect
0811 02:52:31.926314 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0811 02:52:31.927479 (+ 1165us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:52:31.928094 (+ 615us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:52:31.928101 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:52:31.928548 (+ 447us) client_negotiation.cc:189] Negotiated authn=SASL
0811 02:52:31.929241 (+ 693us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:52:31.929259 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:52:31.930167 (+ 908us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:52:31.930171 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:52:31.931054 (+ 883us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:52:31.931063 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:52:31.931576 (+ 513us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:52:31.931580 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:52:31.931749 (+ 169us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:52:31.932389 (+ 640us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 02:52:31.932406 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 02:52:31.933923 (+ 1517us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 02:52:31.935724 (+ 1801us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:52:31.935730 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:52:31.935744 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:52:31.936016 (+ 272us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:52:31.936307 (+ 291us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:52:31.936313 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:52:31.936318 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:52:31.936458 (+ 140us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:52:31.936876 (+ 418us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 02:52:31.936883 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 02:52:31.937225 (+ 342us) client_negotiation.cc:769] Sending connection context
0811 02:52:31.937419 (+ 194us) client_negotiation.cc:240] Negotiation successful
0811 02:52:31.937734 (+ 315us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":261,"thread_start_us":120,"threads_started":1}
WARNING: no policy specified for kudu/127.12.138.195@KRBTEST.COM; defaulting to no policy
I20260811 02:52:31.939951 22893 heartbeater.cc:347] Connected to a master server at 127.12.138.254:34683
I20260811 02:52:31.940282 22893 heartbeater.cc:464] Registering TS with master...
I20260811 02:52:31.940985 22893 heartbeater.cc:511] Master 127.12.138.254:34683 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.138.195@KRBTEST.COM" created.
I20260811 02:52:31.942302 22519 ts_manager.cc:194] Registered new tserver with Master: 737681acde44467ca0932902a31fabc8 (127.12.138.194:41475)
I20260811 02:52:31.943167 22519 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.12.138.194@KRBTEST.COM'} at 127.12.138.194:46921
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.138.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786416707705494-12843-0/krb5kdc/kudu_127.12.138.195.keytab.
Entry for principal kudu/127.12.138.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786416707705494-12843-0/krb5kdc/kudu_127.12.138.195.keytab.
I20260811 02:52:31.960613 12843 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDawEwS/build/debug/bin/kudu
/tmp/dist-test-taskDawEwS/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786416707705494-12843-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786416707705494-12843-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786416707705494-12843-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786416707705494-12843-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.138.195:0
--local_ip_for_outbound_sockets=127.12.138.195
--webserver_interface=127.12.138.195
--webserver_port=0
--tserver_master_addrs=127.12.138.254:34683
--keytab_file=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786416707705494-12843-0/krb5kdc/kudu_127.12.138.195.keytab
--principal=kudu/127.12.138.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.138.212:37315
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786416707705494-12843-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786416707705494-12843-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786416707705494-12843-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 02:52:32.082159 22898 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 02:52:32.082472 22898 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 02:52:32.082566 22898 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 02:52:32.087121 22898 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 02:52:32.087206 22898 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 02:52:32.087316 22898 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.138.195
I20260811 02:52:32.092717 22898 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.138.212:37315
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786416707705494-12843-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786416707705494-12843-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786416707705494-12843-0/krb5kdc/kudu_127.12.138.195.keytab
--principal=kudu/127.12.138.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.138.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786416707705494-12843-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.12.138.195
--webserver_port=0
--tserver_master_addrs=127.12.138.254:34683
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22898
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.138.195
--log_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786416707705494-12843-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 02:09:49 UTC on ccycloud.quasar-fsnqqc.root.comops.site
build id 4579570
I20260811 02:52:32.093888 22898 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 02:52:32.095091 22898 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 02:52:32.105461 22904 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:52:32.105501 22898 server_base.cc:1034] running on GCE node
W20260811 02:52:32.105463 22903 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:52:32.105461 22906 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:52:32.106144 22898 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 02:52:32.106774 22898 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 02:52:32.107986 22898 hybrid_clock.cc:648] HybridClock initialized: now 1786416752107964 us; error 33 us; skew 500 ppm
Aug 11 02:52:32 dist-test-slave-9gcw krb5kdc[22473](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416752, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.138.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 02:52:32.111184 22898 init.cc:377] Logged in from keytab as kudu/127.12.138.195@KRBTEST.COM (short username kudu)
I20260811 02:52:32.112136 22898 webserver.cc:460] Webserver started at http://127.12.138.195:42487/ using document root <none> and password file <none>
I20260811 02:52:32.112713 22898 fs_manager.cc:359] Metadata directory not provided
I20260811 02:52:32.112818 22898 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 02:52:32.113077 22898 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 02:52:32.115196 22898 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786416707705494-12843-0/minicluster-data/ts-2/data/instance:
uuid: "187e9b1be6ae424c894b398121ff7568"
format_stamp: "Formatted at 2026-08-11 02:52:32 on dist-test-slave-9gcw"
server_key: "e1836a14bbffbd14c5c1b84e16bf07ed"
server_key_iv: "067d04bbcf9c292838a86bdcb8373a0d"
server_key_version: "encryptionkey@0"
I20260811 02:52:32.115798 22898 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786416707705494-12843-0/minicluster-data/ts-2/wal/instance:
uuid: "187e9b1be6ae424c894b398121ff7568"
format_stamp: "Formatted at 2026-08-11 02:52:32 on dist-test-slave-9gcw"
server_key: "e1836a14bbffbd14c5c1b84e16bf07ed"
server_key_iv: "067d04bbcf9c292838a86bdcb8373a0d"
server_key_version: "encryptionkey@0"
I20260811 02:52:32.119546 22898 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260811 02:52:32.122056 22914 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 02:52:32.123286 22898 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260811 02:52:32.123421 22898 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786416707705494-12843-0/minicluster-data/ts-2/data,/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786416707705494-12843-0/minicluster-data/ts-2/wal
uuid: "187e9b1be6ae424c894b398121ff7568"
format_stamp: "Formatted at 2026-08-11 02:52:32 on dist-test-slave-9gcw"
server_key: "e1836a14bbffbd14c5c1b84e16bf07ed"
server_key_iv: "067d04bbcf9c292838a86bdcb8373a0d"
server_key_version: "encryptionkey@0"
I20260811 02:52:32.123531 22898 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786416707705494-12843-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786416707705494-12843-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786416707705494-12843-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 02:52:32.141582 22898 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 02:52:32.144994 22898 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 02:52:32.145256 22898 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 02:52:32.145886 22898 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 02:52:32.146821 22898 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 02:52:32.146888 22898 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 02:52:32.146956 22898 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 02:52:32.147003 22898 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 02:52:32.156270 22898 rpc_server.cc:307] RPC server started. Bound to: 127.12.138.195:42717
I20260811 02:52:32.156306 23040 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.138.195:42717 every 8 connection(s)
I20260811 02:52:32.157222 22898 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786416707705494-12843-0/minicluster-data/ts-2/data/info.pb
I20260811 02:52:32.158078 12843 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDawEwS/build/debug/bin/kudu as pid 22898
I20260811 02:52:32.158213 12843 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786416707705494-12843-0/minicluster-data/ts-2/wal/instance
I20260811 02:52:32.158537 12843 external_mini_cluster.cc:1428] Setting key cba9403e91d5973eefeb92643c952dc7
Aug 11 02:52:32 dist-test-slave-9gcw krb5kdc[22473](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416752, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.138.195@KRBTEST.COM for kudu/127.12.138.254@KRBTEST.COM
I20260811 02:52:32.171039 22563 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:52:32.159267 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.138.195:45967 (local address 127.12.138.254:34683)
0811 02:52:32.159448 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0811 02:52:32.159455 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 02:52:32.160434 (+ 979us) server_negotiation.cc:407] Connection header received
0811 02:52:32.161823 (+ 1389us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:52:32.161825 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:52:32.161871 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:52:32.161939 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:52:32.163063 (+ 1124us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:52:32.163618 (+ 555us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:52:32.164743 (+ 1125us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:52:32.164950 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:52:32.165025 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:52:32.167623 (+ 2598us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:52:32.167626 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:52:32.167628 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:52:32.167652 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:52:32.169091 (+ 1439us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:52:32.169725 (+ 634us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:52:32.169728 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:52:32.169729 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:52:32.169812 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:52:32.170104 (+ 292us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:52:32.170106 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:52:32.170107 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:52:32.170302 (+ 195us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:52:32.170380 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0811 02:52:32.170812 (+ 432us) server_negotiation.cc:299] Negotiation successful
0811 02:52:32.170917 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260811 02:52:32.172118 23043 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:52:32.159670 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.138.254:34683 (local address 127.12.138.195:45967)
0811 02:52:32.160257 (+ 587us) negotiation.cc:107] Waiting for socket to connect
0811 02:52:32.160324 (+ 67us) client_negotiation.cc:174] Beginning negotiation
0811 02:52:32.161603 (+ 1279us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:52:32.162076 (+ 473us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:52:32.162083 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:52:32.162448 (+ 365us) client_negotiation.cc:189] Negotiated authn=SASL
0811 02:52:32.162939 (+ 491us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:52:32.162953 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:52:32.163736 (+ 783us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:52:32.163739 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:52:32.164632 (+ 893us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:52:32.164641 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:52:32.165093 (+ 452us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:52:32.165095 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:52:32.165337 (+ 242us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:52:32.165912 (+ 575us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 02:52:32.165928 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0811 02:52:32.167470 (+ 1542us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 02:52:32.169256 (+ 1786us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:52:32.169261 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:52:32.169277 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:52:32.169633 (+ 356us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:52:32.169894 (+ 261us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:52:32.169896 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:52:32.169897 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:52:32.170025 (+ 128us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:52:32.170388 (+ 363us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 02:52:32.170392 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 02:52:32.170609 (+ 217us) client_negotiation.cc:769] Sending connection context
0811 02:52:32.170762 (+ 153us) client_negotiation.cc:240] Negotiation successful
0811 02:52:32.171085 (+ 323us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":374,"thread_start_us":142,"threads_started":1}
I20260811 02:52:32.173398 23041 heartbeater.cc:347] Connected to a master server at 127.12.138.254:34683
I20260811 02:52:32.173681 23041 heartbeater.cc:464] Registering TS with master...
I20260811 02:52:32.174180 23041 heartbeater.cc:511] Master 127.12.138.254:34683 requested a full tablet report, sending...
I20260811 02:52:32.175257 22519 ts_manager.cc:194] Registered new tserver with Master: 187e9b1be6ae424c894b398121ff7568 (127.12.138.195:42717)
I20260811 02:52:32.175803 22519 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.12.138.195@KRBTEST.COM'} at 127.12.138.195:45967
I20260811 02:52:32.184509 12843 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.12.138.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.12.138.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.12.138.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786416707705494-12843-0/krb5kdc/oryx_127.12.138.193.keytab.
Entry for principal oryx/127.12.138.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786416707705494-12843-0/krb5kdc/oryx_127.12.138.193.keytab.
WARNING: no policy specified for oryx/127.12.138.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.12.138.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.12.138.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786416707705494-12843-0/krb5kdc/oryx_127.12.138.194.keytab.
Entry for principal oryx/127.12.138.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786416707705494-12843-0/krb5kdc/oryx_127.12.138.194.keytab.
WARNING: no policy specified for oryx/127.12.138.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.12.138.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.12.138.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786416707705494-12843-0/krb5kdc/oryx_127.12.138.195.keytab.
Entry for principal oryx/127.12.138.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786416707705494-12843-0/krb5kdc/oryx_127.12.138.195.keytab.
I20260811 02:52:32.265647 12843 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDawEwS/build/debug/bin/kudu with pid 22588
I20260811 02:52:32.274950 12843 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDawEwS/build/debug/bin/kudu with pid 22746
I20260811 02:52:32.283147 12843 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDawEwS/build/debug/bin/kudu with pid 22898
I20260811 02:52:32.294123 12843 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDawEwS/build/debug/bin/kudu with pid 22486
I20260811 02:52:32.302636 12843 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDawEwS/build/debug/bin/kudu
/tmp/dist-test-taskDawEwS/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786416707705494-12843-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786416707705494-12843-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786416707705494-12843-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786416707705494-12843-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.138.254:34683
--webserver_interface=127.12.138.254
--webserver_port=36273
--keytab_file=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786416707705494-12843-0/krb5kdc/kudu_127.12.138.254.keytab
--principal=kudu/127.12.138.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.138.212:37315
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.12.138.254:34683
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786416707705494-12843-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786416707705494-12843-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786416707705494-12843-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 02:52:32.417085 23052 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 02:52:32.417433 23052 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 02:52:32.417516 23052 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 02:52:32.421592 23052 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 02:52:32.421681 23052 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 02:52:32.421717 23052 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 02:52:32.421756 23052 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 02:52:32.421782 23052 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 02:52:32.427110 23052 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.138.212:37315
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786416707705494-12843-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786416707705494-12843-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.12.138.254:34683
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786416707705494-12843-0/krb5kdc/kudu_127.12.138.254.keytab
--principal=kudu/127.12.138.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.138.254:34683
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786416707705494-12843-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.12.138.254
--webserver_port=36273
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23052
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786416707705494-12843-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 02:09:49 UTC on ccycloud.quasar-fsnqqc.root.comops.site
build id 4579570
I20260811 02:52:32.428329 23052 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 02:52:32.429484 23052 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260811 02:52:32.438815 23052 server_base.cc:1034] running on GCE node
W20260811 02:52:32.438697 23058 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:52:32.438764 23057 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:52:32.438697 23060 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:52:32.439703 23052 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 02:52:32.440928 23052 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 02:52:32.442128 23052 hybrid_clock.cc:648] HybridClock initialized: now 1786416752442118 us; error 67 us; skew 500 ppm
Aug 11 02:52:32 dist-test-slave-9gcw krb5kdc[22473](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416752, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.138.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 02:52:32.445358 23052 init.cc:377] Logged in from keytab as kudu/127.12.138.254@KRBTEST.COM (short username kudu)
I20260811 02:52:32.446507 23052 webserver.cc:460] Webserver started at http://127.12.138.254:36273/ using document root <none> and password file <none>
I20260811 02:52:32.447170 23052 fs_manager.cc:359] Metadata directory not provided
I20260811 02:52:32.447299 23052 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 02:52:32.451577 23052 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260811 02:52:32.454360 23069 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 02:52:32.455753 23052 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 02:52:32.455940 23052 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786416707705494-12843-0/minicluster-data/master-0/data,/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786416707705494-12843-0/minicluster-data/master-0/wal
uuid: "b18a0d21a30e49938b995a4676caaad8"
format_stamp: "Formatted at 2026-08-11 02:52:31 on dist-test-slave-9gcw"
server_key: "c7546f4b32447ef65eba18e28fe36f26"
server_key_iv: "1909c7f5778486e0303a15fc296e0df0"
server_key_version: "encryptionkey@0"
I20260811 02:52:32.456578 23052 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786416707705494-12843-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786416707705494-12843-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786416707705494-12843-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 02:52:32.489919 23052 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 02:52:32.493132 23052 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 02:52:32.493429 23052 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 02:52:32.501232 23052 rpc_server.cc:307] RPC server started. Bound to: 127.12.138.254:34683
I20260811 02:52:32.501278 23131 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.138.254:34683 every 8 connection(s)
I20260811 02:52:32.502339 23052 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786416707705494-12843-0/minicluster-data/master-0/data/info.pb
I20260811 02:52:32.506661 23133 sys_catalog.cc:263] Verifying existing consensus state
I20260811 02:52:32.508351 23133 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b18a0d21a30e49938b995a4676caaad8: Bootstrap starting.
I20260811 02:52:32.510339 12843 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDawEwS/build/debug/bin/kudu as pid 23052
I20260811 02:52:32.510893 12843 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDawEwS/build/debug/bin/kudu
/tmp/dist-test-taskDawEwS/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786416707705494-12843-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786416707705494-12843-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786416707705494-12843-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786416707705494-12843-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.138.193:36935
--local_ip_for_outbound_sockets=127.12.138.193
--tserver_master_addrs=127.12.138.254:34683
--webserver_port=41325
--webserver_interface=127.12.138.193
--keytab_file=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786416707705494-12843-0/krb5kdc/kudu_127.12.138.193.keytab
--principal=kudu/127.12.138.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.138.212:37315
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.12.138.193
--keytab_file=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786416707705494-12843-0/krb5kdc/oryx_127.12.138.193.keytab with env {KRB5CCNAME=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786416707705494-12843-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786416707705494-12843-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786416707705494-12843-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 02:52:32.526041 23133 log.cc:824] T 00000000000000000000000000000000 P b18a0d21a30e49938b995a4676caaad8: Log is configured to *not* fsync() on all Append() calls
I20260811 02:52:32.532598 23133 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b18a0d21a30e49938b995a4676caaad8: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 02:52:32.533000 23133 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b18a0d21a30e49938b995a4676caaad8: Bootstrap complete.
I20260811 02:52:32.537420 23133 raft_consensus.cc:348] T 00000000000000000000000000000000 P b18a0d21a30e49938b995a4676caaad8 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b18a0d21a30e49938b995a4676caaad8" member_type: VOTER last_known_addr { host: "127.12.138.254" port: 34683 } }
I20260811 02:52:32.537823 23133 raft_consensus.cc:729] T 00000000000000000000000000000000 P b18a0d21a30e49938b995a4676caaad8 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: b18a0d21a30e49938b995a4676caaad8, State: Initialized, Role: FOLLOWER
I20260811 02:52:32.538373 23133 consensus_queue.cc:260] T 00000000000000000000000000000000 P b18a0d21a30e49938b995a4676caaad8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b18a0d21a30e49938b995a4676caaad8" member_type: VOTER last_known_addr { host: "127.12.138.254" port: 34683 } }
I20260811 02:52:32.538498 23133 raft_consensus.cc:388] T 00000000000000000000000000000000 P b18a0d21a30e49938b995a4676caaad8 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 02:52:32.538585 23133 raft_consensus.cc:482] T 00000000000000000000000000000000 P b18a0d21a30e49938b995a4676caaad8 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 02:52:32.538704 23133 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b18a0d21a30e49938b995a4676caaad8 [term 1 FOLLOWER]: Advancing to term 2
I20260811 02:52:32.540614 23133 raft_consensus.cc:504] T 00000000000000000000000000000000 P b18a0d21a30e49938b995a4676caaad8 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b18a0d21a30e49938b995a4676caaad8" member_type: VOTER last_known_addr { host: "127.12.138.254" port: 34683 } }
I20260811 02:52:32.540947 23133 leader_election.cc:302] T 00000000000000000000000000000000 P b18a0d21a30e49938b995a4676caaad8 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b18a0d21a30e49938b995a4676caaad8; no voters:
I20260811 02:52:32.541323 23133 leader_election.cc:288] T 00000000000000000000000000000000 P b18a0d21a30e49938b995a4676caaad8 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260811 02:52:32.541416 23137 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b18a0d21a30e49938b995a4676caaad8 [term 2 FOLLOWER]: Leader election won for term 2
I20260811 02:52:32.541625 23137 raft_consensus.cc:686] T 00000000000000000000000000000000 P b18a0d21a30e49938b995a4676caaad8 [term 2 LEADER]: Becoming Leader. State: Replica: b18a0d21a30e49938b995a4676caaad8, State: Running, Role: LEADER
I20260811 02:52:32.541903 23137 consensus_queue.cc:237] T 00000000000000000000000000000000 P b18a0d21a30e49938b995a4676caaad8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b18a0d21a30e49938b995a4676caaad8" member_type: VOTER last_known_addr { host: "127.12.138.254" port: 34683 } }
I20260811 02:52:32.542536 23133 sys_catalog.cc:564] T 00000000000000000000000000000000 P b18a0d21a30e49938b995a4676caaad8 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 02:52:32.542820 23137 sys_catalog.cc:455] T 00000000000000000000000000000000 P b18a0d21a30e49938b995a4676caaad8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b18a0d21a30e49938b995a4676caaad8. Latest consensus state: current_term: 2 leader_uuid: "b18a0d21a30e49938b995a4676caaad8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b18a0d21a30e49938b995a4676caaad8" member_type: VOTER last_known_addr { host: "127.12.138.254" port: 34683 } } }
I20260811 02:52:32.542940 23137 sys_catalog.cc:458] T 00000000000000000000000000000000 P b18a0d21a30e49938b995a4676caaad8 [sys.catalog]: This master's current role is: LEADER
I20260811 02:52:32.543310 23141 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 02:52:32.543352 23138 sys_catalog.cc:455] T 00000000000000000000000000000000 P b18a0d21a30e49938b995a4676caaad8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "b18a0d21a30e49938b995a4676caaad8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b18a0d21a30e49938b995a4676caaad8" member_type: VOTER last_known_addr { host: "127.12.138.254" port: 34683 } } }
I20260811 02:52:32.543568 23138 sys_catalog.cc:458] T 00000000000000000000000000000000 P b18a0d21a30e49938b995a4676caaad8 [sys.catalog]: This master's current role is: LEADER
I20260811 02:52:32.546634 23141 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 02:52:32.547277 23141 catalog_manager.cc:1259] Loaded cluster ID: dfaaf1425509400a98919b096d81a016
I20260811 02:52:32.547371 23141 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 02:52:32.550933 23141 catalog_manager.cc:1504] Loading token signing keys...
I20260811 02:52:32.551616 23141 catalog_manager.cc:5879] T 00000000000000000000000000000000 P b18a0d21a30e49938b995a4676caaad8: Loaded TSK: 0
I20260811 02:52:32.552842 23141 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260811 02:52:32.629765 23135 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 02:52:32.630074 23135 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 02:52:32.630155 23135 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 02:52:32.634351 23135 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 02:52:32.634450 23135 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 02:52:32.634548 23135 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.138.193
I20260811 02:52:32.640357 23135 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.138.212:37315
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786416707705494-12843-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786416707705494-12843-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786416707705494-12843-0/krb5kdc/oryx_127.12.138.193.keytab
--principal=oryx/127.12.138.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.138.193:36935
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786416707705494-12843-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.12.138.193
--webserver_port=41325
--tserver_master_addrs=127.12.138.254:34683
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23135
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.138.193
--log_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786416707705494-12843-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 02:09:49 UTC on ccycloud.quasar-fsnqqc.root.comops.site
build id 4579570
I20260811 02:52:32.641603 23135 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 02:52:32.642763 23135 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 02:52:32.653374 23163 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:52:32.653388 23165 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:52:32.653388 23162 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:52:32.653934 23135 server_base.cc:1034] running on GCE node
I20260811 02:52:32.654479 23135 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 02:52:32.655364 23135 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 02:52:32.656591 23135 hybrid_clock.cc:648] HybridClock initialized: now 1786416752656549 us; error 49 us; skew 500 ppm
Aug 11 02:52:32 dist-test-slave-9gcw krb5kdc[22473](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416752, etypes {rep=17 tkt=17 ses=17}, oryx/127.12.138.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 02:52:32.660306 23135 init.cc:377] Logged in from keytab as oryx/127.12.138.193@KRBTEST.COM (short username oryx)
I20260811 02:52:32.661428 23135 webserver.cc:460] Webserver started at http://127.12.138.193:41325/ using document root <none> and password file <none>
I20260811 02:52:32.662076 23135 fs_manager.cc:359] Metadata directory not provided
I20260811 02:52:32.662158 23135 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 02:52:32.666268 23135 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.001s
I20260811 02:52:32.668681 23172 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 02:52:32.670074 23135 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260811 02:52:32.670228 23135 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786416707705494-12843-0/minicluster-data/ts-0/data,/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786416707705494-12843-0/minicluster-data/ts-0/wal
uuid: "d3a1386780b941a6b10f550a18c16fb9"
format_stamp: "Formatted at 2026-08-11 02:52:31 on dist-test-slave-9gcw"
server_key: "e10f6a7896ef05f7dc9db876f8142907"
server_key_iv: "fabe4486b36618567fa9fe9a00e19bf6"
server_key_version: "encryptionkey@0"
I20260811 02:52:32.670835 23135 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786416707705494-12843-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786416707705494-12843-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786416707705494-12843-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 02:52:32.685881 23135 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 02:52:32.689405 23135 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 02:52:32.689621 23135 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 02:52:32.690277 23135 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 02:52:32.691159 23135 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 02:52:32.691206 23135 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 02:52:32.691272 23135 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 02:52:32.691304 23135 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 02:52:32.700076 23135 rpc_server.cc:307] RPC server started. Bound to: 127.12.138.193:36935
I20260811 02:52:32.700182 23309 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.138.193:36935 every 8 connection(s)
I20260811 02:52:32.701210 23135 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786416707705494-12843-0/minicluster-data/ts-0/data/info.pb
I20260811 02:52:32.708835 12843 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDawEwS/build/debug/bin/kudu as pid 23135
I20260811 02:52:32.709362 12843 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDawEwS/build/debug/bin/kudu
/tmp/dist-test-taskDawEwS/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786416707705494-12843-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786416707705494-12843-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786416707705494-12843-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786416707705494-12843-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.138.194:41475
--local_ip_for_outbound_sockets=127.12.138.194
--tserver_master_addrs=127.12.138.254:34683
--webserver_port=37063
--webserver_interface=127.12.138.194
--keytab_file=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786416707705494-12843-0/krb5kdc/kudu_127.12.138.194.keytab
--principal=kudu/127.12.138.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.138.212:37315
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.12.138.194
--keytab_file=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786416707705494-12843-0/krb5kdc/oryx_127.12.138.194.keytab with env {KRB5CCNAME=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786416707705494-12843-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786416707705494-12843-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786416707705494-12843-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 11 02:52:32 dist-test-slave-9gcw krb5kdc[22473](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.138.193@KRBTEST.COM for oryx/127.12.138.254@KRBTEST.COM, Server not found in Kerberos database
Aug 11 02:52:32 dist-test-slave-9gcw krb5kdc[22473](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.138.193@KRBTEST.COM for oryx/127.12.138.254@KRBTEST.COM, Server not found in Kerberos database
W20260811 02:52:32.714984 23312 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 02:52:32.703442 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.138.193:39409 (local address 127.12.138.254:34683)
0811 02:52:32.703977 (+ 535us) server_negotiation.cc:206] Beginning negotiation
0811 02:52:32.703997 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0811 02:52:32.704210 (+ 213us) server_negotiation.cc:407] Connection header received
0811 02:52:32.705751 (+ 1541us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:52:32.705763 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:52:32.706080 (+ 317us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:52:32.706434 (+ 354us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:52:32.707589 (+ 1155us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:52:32.708588 (+ 999us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:52:32.710048 (+ 1460us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:52:32.710543 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:52:32.710731 (+ 188us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:52:32.714255 (+ 3524us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.12.138.193:39409: BlockingRecv error: recv got EOF from 127.12.138.193:39409 (error 108)
Metrics: {"server-negotiator.queue_time_us":365,"thread_start_us":141,"threads_started":1}
I20260811 02:52:32.714994 23313 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:52:32.703512 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.138.254:34683 (local address 127.12.138.193:39409)
0811 02:52:32.704033 (+ 521us) negotiation.cc:107] Waiting for socket to connect
0811 02:52:32.704089 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0811 02:52:32.705325 (+ 1236us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:52:32.706494 (+ 1169us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:52:32.706502 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:52:32.706945 (+ 443us) client_negotiation.cc:189] Negotiated authn=SASL
0811 02:52:32.707408 (+ 463us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:52:32.707421 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:52:32.708741 (+ 1320us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:52:32.708746 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:52:32.709891 (+ 1145us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:52:32.709902 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:52:32.710717 (+ 815us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:52:32.710721 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:52:32.710929 (+ 208us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:52:32.711558 (+ 629us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 02:52:32.711574 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0811 02:52:32.714032 (+ 2458us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.12.138.254:34683: Server oryx/127.12.138.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":305,"thread_start_us":148,"threads_started":1}
W20260811 02:52:32.715770 23310 heartbeater.cc:644] Failed to heartbeat to 127.12.138.254:34683 (0 consecutive failures): Not authorized: Failed to ping master at 127.12.138.254:34683: Client connection negotiation failed: client connection to 127.12.138.254:34683: Server oryx/127.12.138.254@KRBTEST.COM not found in Kerberos database
Aug 11 02:52:32 dist-test-slave-9gcw krb5kdc[22473](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.138.193@KRBTEST.COM for oryx/127.12.138.254@KRBTEST.COM, Server not found in Kerberos database
Aug 11 02:52:32 dist-test-slave-9gcw krb5kdc[22473](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.138.193@KRBTEST.COM for oryx/127.12.138.254@KRBTEST.COM, Server not found in Kerberos database
I20260811 02:52:32.722285 23313 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:52:32.716290 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.138.254:34683 (local address 127.12.138.193:33693)
0811 02:52:32.716499 (+ 209us) negotiation.cc:107] Waiting for socket to connect
0811 02:52:32.716515 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 02:52:32.716616 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:52:32.716889 (+ 273us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:52:32.716894 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:52:32.717083 (+ 189us) client_negotiation.cc:189] Negotiated authn=SASL
0811 02:52:32.717260 (+ 177us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:52:32.717270 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:52:32.717992 (+ 722us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:52:32.717999 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:52:32.719021 (+ 1022us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:52:32.719035 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:52:32.719469 (+ 434us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:52:32.719476 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:52:32.719604 (+ 128us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:52:32.720106 (+ 502us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 02:52:32.720124 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0811 02:52:32.722151 (+ 2027us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.12.138.254:34683: Server oryx/127.12.138.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":120}
W20260811 02:52:32.722419 23312 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 02:52:32.716382 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.138.193:33693 (local address 127.12.138.254:34683)
0811 02:52:32.716496 (+ 114us) server_negotiation.cc:206] Beginning negotiation
0811 02:52:32.716499 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 02:52:32.716566 (+ 67us) server_negotiation.cc:407] Connection header received
0811 02:52:32.716711 (+ 145us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:52:32.716713 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:52:32.716755 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:52:32.716835 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:52:32.717380 (+ 545us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:52:32.717879 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:52:32.719148 (+ 1269us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:52:32.719354 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:52:32.719431 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:52:32.722281 (+ 2850us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.12.138.193:33693: BlockingRecv error: recv got EOF from 127.12.138.193:33693 (error 108)
Metrics: {"server-negotiator.queue_time_us":42}
Aug 11 02:52:32 dist-test-slave-9gcw krb5kdc[22473](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.138.193@KRBTEST.COM for oryx/127.12.138.254@KRBTEST.COM, Server not found in Kerberos database
Aug 11 02:52:32 dist-test-slave-9gcw krb5kdc[22473](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.138.193@KRBTEST.COM for oryx/127.12.138.254@KRBTEST.COM, Server not found in Kerberos database
I20260811 02:52:32.727914 23313 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:52:32.722858 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.138.254:34683 (local address 127.12.138.193:46383)
0811 02:52:32.722976 (+ 118us) negotiation.cc:107] Waiting for socket to connect
0811 02:52:32.722992 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 02:52:32.723065 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:52:32.723317 (+ 252us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:52:32.723319 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:52:32.723442 (+ 123us) client_negotiation.cc:189] Negotiated authn=SASL
0811 02:52:32.723509 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:52:32.723514 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:52:32.724537 (+ 1023us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:52:32.724540 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:52:32.725042 (+ 502us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:52:32.725050 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:52:32.725608 (+ 558us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:52:32.725611 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:52:32.725669 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:52:32.726142 (+ 473us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 02:52:32.726153 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0811 02:52:32.727823 (+ 1670us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.12.138.254:34683: Server oryx/127.12.138.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":50}
W20260811 02:52:32.728130 23310 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260811 02:52:32.728361 23312 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 02:52:32.722921 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.138.193:46383 (local address 127.12.138.254:34683)
0811 02:52:32.723030 (+ 109us) server_negotiation.cc:206] Beginning negotiation
0811 02:52:32.723034 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 02:52:32.723049 (+ 15us) server_negotiation.cc:407] Connection header received
0811 02:52:32.723150 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:52:32.723154 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:52:32.723201 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:52:32.723284 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:52:32.723609 (+ 325us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:52:32.724425 (+ 816us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:52:32.725166 (+ 741us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:52:32.725504 (+ 338us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:52:32.725606 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:52:32.728005 (+ 2399us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.12.138.193:46383: BlockingRecv error: recv got EOF from 127.12.138.193:46383 (error 108)
Metrics: {"server-negotiator.queue_time_us":45}
W20260811 02:52:32.826923 23314 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 02:52:32.827198 23314 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 02:52:32.827239 23314 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 02:52:32.831169 23314 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 02:52:32.831224 23314 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 02:52:32.831279 23314 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.138.194
I20260811 02:52:32.836475 23314 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.138.212:37315
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786416707705494-12843-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786416707705494-12843-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786416707705494-12843-0/krb5kdc/oryx_127.12.138.194.keytab
--principal=oryx/127.12.138.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.138.194:41475
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786416707705494-12843-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.12.138.194
--webserver_port=37063
--tserver_master_addrs=127.12.138.254:34683
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23314
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.138.194
--log_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786416707705494-12843-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 02:09:49 UTC on ccycloud.quasar-fsnqqc.root.comops.site
build id 4579570
I20260811 02:52:32.837505 23314 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 02:52:32.838480 23314 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 02:52:32.848477 23320 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:52:32.850790 23322 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:52:32.851039 23319 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:52:32.851001 23314 server_base.cc:1034] running on GCE node
I20260811 02:52:32.851449 23314 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 02:52:32.852092 23314 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 02:52:32.853264 23314 hybrid_clock.cc:648] HybridClock initialized: now 1786416752853251 us; error 27 us; skew 500 ppm
Aug 11 02:52:32 dist-test-slave-9gcw krb5kdc[22473](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416752, etypes {rep=17 tkt=17 ses=17}, oryx/127.12.138.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 02:52:32.856084 23314 init.cc:377] Logged in from keytab as oryx/127.12.138.194@KRBTEST.COM (short username oryx)
I20260811 02:52:32.857012 23314 webserver.cc:460] Webserver started at http://127.12.138.194:37063/ using document root <none> and password file <none>
I20260811 02:52:32.857641 23314 fs_manager.cc:359] Metadata directory not provided
I20260811 02:52:32.857757 23314 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 02:52:32.861685 23314 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260811 02:52:32.863830 23332 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 02:52:32.865348 23314 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 02:52:32.865479 23314 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786416707705494-12843-0/minicluster-data/ts-1/data,/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786416707705494-12843-0/minicluster-data/ts-1/wal
uuid: "737681acde44467ca0932902a31fabc8"
format_stamp: "Formatted at 2026-08-11 02:52:31 on dist-test-slave-9gcw"
server_key: "844e47d595cab1b56681859b29f1576f"
server_key_iv: "c6d1187d3b7433dd393b38f4b23e1646"
server_key_version: "encryptionkey@0"
I20260811 02:52:32.866046 23314 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786416707705494-12843-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786416707705494-12843-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786416707705494-12843-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 02:52:32.896281 23314 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 02:52:32.899972 23314 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 02:52:32.900202 23314 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 02:52:32.900861 23314 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 02:52:32.901760 23314 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 02:52:32.901842 23314 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 02:52:32.901916 23314 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 02:52:32.901959 23314 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 02:52:32.911762 23314 rpc_server.cc:307] RPC server started. Bound to: 127.12.138.194:41475
I20260811 02:52:32.911798 23465 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.138.194:41475 every 8 connection(s)
I20260811 02:52:32.912818 23314 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786416707705494-12843-0/minicluster-data/ts-1/data/info.pb
I20260811 02:52:32.917492 12843 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDawEwS/build/debug/bin/kudu as pid 23314
I20260811 02:52:32.917959 12843 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDawEwS/build/debug/bin/kudu
/tmp/dist-test-taskDawEwS/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786416707705494-12843-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786416707705494-12843-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786416707705494-12843-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786416707705494-12843-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.138.195:42717
--local_ip_for_outbound_sockets=127.12.138.195
--tserver_master_addrs=127.12.138.254:34683
--webserver_port=42487
--webserver_interface=127.12.138.195
--keytab_file=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786416707705494-12843-0/krb5kdc/kudu_127.12.138.195.keytab
--principal=kudu/127.12.138.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.138.212:37315
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.12.138.195
--keytab_file=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786416707705494-12843-0/krb5kdc/oryx_127.12.138.195.keytab with env {KRB5CCNAME=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786416707705494-12843-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786416707705494-12843-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786416707705494-12843-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 11 02:52:32 dist-test-slave-9gcw krb5kdc[22473](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.138.194@KRBTEST.COM for oryx/127.12.138.254@KRBTEST.COM, Server not found in Kerberos database
Aug 11 02:52:32 dist-test-slave-9gcw krb5kdc[22473](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.138.194@KRBTEST.COM for oryx/127.12.138.254@KRBTEST.COM, Server not found in Kerberos database
W20260811 02:52:32.925654 23312 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 02:52:32.914837 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.138.194:42601 (local address 127.12.138.254:34683)
0811 02:52:32.915015 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0811 02:52:32.915020 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 02:52:32.915846 (+ 826us) server_negotiation.cc:407] Connection header received
0811 02:52:32.917105 (+ 1259us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:52:32.917108 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:52:32.917224 (+ 116us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:52:32.917341 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:52:32.918513 (+ 1172us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:52:32.919349 (+ 836us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:52:32.920490 (+ 1141us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:52:32.920877 (+ 387us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:52:32.921008 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:52:32.925388 (+ 4380us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.12.138.194:42601: BlockingRecv error: recv got EOF from 127.12.138.194:42601 (error 108)
Metrics: {"server-negotiator.queue_time_us":70}
I20260811 02:52:32.926271 23469 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:52:32.915213 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.138.254:34683 (local address 127.12.138.194:42601)
0811 02:52:32.915677 (+ 464us) negotiation.cc:107] Waiting for socket to connect
0811 02:52:32.915719 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0811 02:52:32.916896 (+ 1177us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:52:32.917491 (+ 595us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:52:32.917498 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:52:32.917873 (+ 375us) client_negotiation.cc:189] Negotiated authn=SASL
0811 02:52:32.918357 (+ 484us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:52:32.918370 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:52:32.919478 (+ 1108us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:52:32.919480 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:52:32.920359 (+ 879us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:52:32.920368 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:52:32.921029 (+ 661us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:52:32.921031 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:52:32.921255 (+ 224us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:52:32.921895 (+ 640us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 02:52:32.921911 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0811 02:52:32.925076 (+ 3165us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.12.138.254:34683: Server oryx/127.12.138.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":274,"thread_start_us":121,"threads_started":1}
W20260811 02:52:32.927232 23467 heartbeater.cc:644] Failed to heartbeat to 127.12.138.254:34683 (0 consecutive failures): Not authorized: Failed to ping master at 127.12.138.254:34683: Client connection negotiation failed: client connection to 127.12.138.254:34683: Server oryx/127.12.138.254@KRBTEST.COM not found in Kerberos database
Aug 11 02:52:32 dist-test-slave-9gcw krb5kdc[22473](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.138.194@KRBTEST.COM for oryx/127.12.138.254@KRBTEST.COM, Server not found in Kerberos database
Aug 11 02:52:32 dist-test-slave-9gcw krb5kdc[22473](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.138.194@KRBTEST.COM for oryx/127.12.138.254@KRBTEST.COM, Server not found in Kerberos database
I20260811 02:52:32.934316 23469 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:52:32.927783 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.138.254:34683 (local address 127.12.138.194:38325)
0811 02:52:32.928093 (+ 310us) negotiation.cc:107] Waiting for socket to connect
0811 02:52:32.928133 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0811 02:52:32.928251 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:52:32.928720 (+ 469us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:52:32.928724 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:52:32.928973 (+ 249us) client_negotiation.cc:189] Negotiated authn=SASL
0811 02:52:32.929096 (+ 123us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:52:32.929104 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:52:32.930499 (+ 1395us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:52:32.930502 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:52:32.931046 (+ 544us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:52:32.931054 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:52:32.931681 (+ 627us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:52:32.931684 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:52:32.931752 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:52:32.932301 (+ 549us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 02:52:32.932312 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0811 02:52:32.934201 (+ 1889us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.12.138.254:34683: Server oryx/127.12.138.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":188}
W20260811 02:52:32.934760 23312 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 02:52:32.928028 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.138.194:38325 (local address 127.12.138.254:34683)
0811 02:52:32.928170 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0811 02:52:32.928176 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 02:52:32.928194 (+ 18us) server_negotiation.cc:407] Connection header received
0811 02:52:32.928451 (+ 257us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:52:32.928456 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:52:32.928548 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:52:32.928650 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:52:32.929288 (+ 638us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:52:32.930374 (+ 1086us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:52:32.931166 (+ 792us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:52:32.931565 (+ 399us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:52:32.931691 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:52:32.934356 (+ 2665us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.12.138.194:38325: BlockingRecv error: recv got EOF from 127.12.138.194:38325 (error 108)
Metrics: {"server-negotiator.queue_time_us":53}
Aug 11 02:52:32 dist-test-slave-9gcw krb5kdc[22473](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.138.194@KRBTEST.COM for oryx/127.12.138.254@KRBTEST.COM, Server not found in Kerberos database
Aug 11 02:52:32 dist-test-slave-9gcw krb5kdc[22473](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.138.194@KRBTEST.COM for oryx/127.12.138.254@KRBTEST.COM, Server not found in Kerberos database
I20260811 02:52:32.940076 23469 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:52:32.934899 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.138.254:34683 (local address 127.12.138.194:54575)
0811 02:52:32.935088 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0811 02:52:32.935103 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 02:52:32.935178 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:52:32.935411 (+ 233us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:52:32.935413 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:52:32.935537 (+ 124us) client_negotiation.cc:189] Negotiated authn=SASL
0811 02:52:32.935604 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:52:32.935608 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:52:32.936568 (+ 960us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:52:32.936570 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:52:32.937065 (+ 495us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:52:32.937071 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:52:32.937620 (+ 549us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:52:32.937623 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:52:32.937680 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:52:32.938149 (+ 469us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 02:52:32.938160 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0811 02:52:32.939992 (+ 1832us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.12.138.254:34683: Server oryx/127.12.138.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":102}
W20260811 02:52:32.940250 23312 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 02:52:32.934933 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.138.194:54575 (local address 127.12.138.254:34683)
0811 02:52:32.935181 (+ 248us) server_negotiation.cc:206] Beginning negotiation
0811 02:52:32.935185 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 02:52:32.935200 (+ 15us) server_negotiation.cc:407] Connection header received
0811 02:52:32.935257 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:52:32.935260 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:52:32.935299 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:52:32.935386 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:52:32.935705 (+ 319us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:52:32.936470 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:52:32.937187 (+ 717us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:52:32.937520 (+ 333us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:52:32.937620 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:52:32.940137 (+ 2517us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.12.138.194:54575: BlockingRecv error: recv got EOF from 127.12.138.194:54575 (error 108)
Metrics: {"server-negotiator.queue_time_us":194,"thread_start_us":110,"threads_started":1}
W20260811 02:52:32.940470 23467 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260811 02:52:33.040359 23470 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 02:52:33.040652 23470 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 02:52:33.040724 23470 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 02:52:33.045420 23470 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 02:52:33.045485 23470 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 02:52:33.045590 23470 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.138.195
I20260811 02:52:33.051540 23470 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.138.212:37315
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786416707705494-12843-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786416707705494-12843-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786416707705494-12843-0/krb5kdc/oryx_127.12.138.195.keytab
--principal=oryx/127.12.138.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.138.195:42717
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786416707705494-12843-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.12.138.195
--webserver_port=42487
--tserver_master_addrs=127.12.138.254:34683
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23470
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.138.195
--log_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786416707705494-12843-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 02:09:49 UTC on ccycloud.quasar-fsnqqc.root.comops.site
build id 4579570
I20260811 02:52:33.052713 23470 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 02:52:33.053913 23470 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 02:52:33.064122 23477 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:52:33.064109 23480 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:52:33.064162 23478 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:52:33.064401 23470 server_base.cc:1034] running on GCE node
I20260811 02:52:33.064973 23470 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 02:52:33.065747 23470 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 02:52:33.066939 23470 hybrid_clock.cc:648] HybridClock initialized: now 1786416753066923 us; error 29 us; skew 500 ppm
Aug 11 02:52:33 dist-test-slave-9gcw krb5kdc[22473](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416753, etypes {rep=17 tkt=17 ses=17}, oryx/127.12.138.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 02:52:33.069820 23470 init.cc:377] Logged in from keytab as oryx/127.12.138.195@KRBTEST.COM (short username oryx)
I20260811 02:52:33.070772 23470 webserver.cc:460] Webserver started at http://127.12.138.195:42487/ using document root <none> and password file <none>
I20260811 02:52:33.071375 23470 fs_manager.cc:359] Metadata directory not provided
I20260811 02:52:33.071497 23470 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 02:52:33.078933 23470 fs_manager.cc:709] Time spent opening directory manager: real 0.006s user 0.002s sys 0.000s
I20260811 02:52:33.081710 23487 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 02:52:33.082947 23470 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 02:52:33.083096 23470 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786416707705494-12843-0/minicluster-data/ts-2/data,/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786416707705494-12843-0/minicluster-data/ts-2/wal
uuid: "187e9b1be6ae424c894b398121ff7568"
format_stamp: "Formatted at 2026-08-11 02:52:32 on dist-test-slave-9gcw"
server_key: "e1836a14bbffbd14c5c1b84e16bf07ed"
server_key_iv: "067d04bbcf9c292838a86bdcb8373a0d"
server_key_version: "encryptionkey@0"
I20260811 02:52:33.083662 23470 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786416707705494-12843-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786416707705494-12843-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786416707705494-12843-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 02:52:33.103212 23470 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 02:52:33.107121 23470 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 02:52:33.107388 23470 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 02:52:33.108103 23470 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 02:52:33.109117 23470 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 02:52:33.109263 23470 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 02:52:33.109349 23470 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 02:52:33.109404 23470 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 02:52:33.119180 23470 rpc_server.cc:307] RPC server started. Bound to: 127.12.138.195:42717
I20260811 02:52:33.119241 23620 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.138.195:42717 every 8 connection(s)
I20260811 02:52:33.120369 23470 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786416707705494-12843-0/minicluster-data/ts-2/data/info.pb
I20260811 02:52:33.121397 12843 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDawEwS/build/debug/bin/kudu as pid 23470
Aug 11 02:52:33 dist-test-slave-9gcw krb5kdc[22473](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.138.195@KRBTEST.COM for oryx/127.12.138.254@KRBTEST.COM, Server not found in Kerberos database
I20260811 02:52:33.131623 23312 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:52:33.122213 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38846 (local address 127.12.138.254:34683)
0811 02:52:33.122366 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0811 02:52:33.122371 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 02:52:33.122657 (+ 286us) server_negotiation.cc:407] Connection header received
0811 02:52:33.122811 (+ 154us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:52:33.122815 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:52:33.122872 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:52:33.122971 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:52:33.123819 (+ 848us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:52:33.124450 (+ 631us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:52:33.125442 (+ 992us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:52:33.125696 (+ 254us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:52:33.125815 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:52:33.127421 (+ 1606us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:52:33.127426 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:52:33.127440 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:52:33.127468 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:52:33.129384 (+ 1916us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:52:33.129847 (+ 463us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:52:33.129852 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:52:33.129857 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:52:33.129972 (+ 115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:52:33.130275 (+ 303us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:52:33.130279 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:52:33.130281 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:52:33.130616 (+ 335us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:52:33.130836 (+ 220us) server_negotiation.cc:1035] Waiting for connection context
0811 02:52:33.131214 (+ 378us) server_negotiation.cc:299] Negotiation successful
0811 02:52:33.131481 (+ 267us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
Aug 11 02:52:33 dist-test-slave-9gcw krb5kdc[22473](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.138.195@KRBTEST.COM for oryx/127.12.138.254@KRBTEST.COM, Server not found in Kerberos database
W20260811 02:52:33.133141 23471 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 02:52:33.123053 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.138.195:47995 (local address 127.12.138.254:34683)
0811 02:52:33.123175 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0811 02:52:33.123180 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 02:52:33.123950 (+ 770us) server_negotiation.cc:407] Connection header received
0811 02:52:33.125477 (+ 1527us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:52:33.125481 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:52:33.125537 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:52:33.125641 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:52:33.127063 (+ 1422us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:52:33.127652 (+ 589us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:52:33.128925 (+ 1273us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:52:33.129248 (+ 323us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:52:33.129370 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:52:33.133061 (+ 3691us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.12.138.195:47995: BlockingRecv error: recv got EOF from 127.12.138.195:47995 (error 108)
Metrics: {"server-negotiator.queue_time_us":36}
I20260811 02:52:33.133996 23624 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:52:33.123240 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.138.254:34683 (local address 127.12.138.195:47995)
0811 02:52:33.123786 (+ 546us) negotiation.cc:107] Waiting for socket to connect
0811 02:52:33.123843 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0811 02:52:33.125258 (+ 1415us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:52:33.125825 (+ 567us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:52:33.125832 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:52:33.126308 (+ 476us) client_negotiation.cc:189] Negotiated authn=SASL
0811 02:52:33.126904 (+ 596us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:52:33.126919 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:52:33.127794 (+ 875us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:52:33.127798 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:52:33.128776 (+ 978us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:52:33.128785 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:52:33.129410 (+ 625us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:52:33.129415 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:52:33.129600 (+ 185us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:52:33.130196 (+ 596us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 02:52:33.130213 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 02:52:33.132970 (+ 2757us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.12.138.254:34683: Server oryx/127.12.138.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":314,"thread_start_us":161,"threads_started":1}
W20260811 02:52:33.134577 23621 heartbeater.cc:644] Failed to heartbeat to 127.12.138.254:34683 (0 consecutive failures): Not authorized: Failed to ping master at 127.12.138.254:34683: Client connection negotiation failed: client connection to 127.12.138.254:34683: Server oryx/127.12.138.254@KRBTEST.COM not found in Kerberos database
Aug 11 02:52:33 dist-test-slave-9gcw krb5kdc[22473](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.138.195@KRBTEST.COM for oryx/127.12.138.254@KRBTEST.COM, Server not found in Kerberos database
Aug 11 02:52:33 dist-test-slave-9gcw krb5kdc[22473](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.138.195@KRBTEST.COM for oryx/127.12.138.254@KRBTEST.COM, Server not found in Kerberos database
I20260811 02:52:33.140480 23624 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:52:33.135031 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.138.254:34683 (local address 127.12.138.195:43899)
0811 02:52:33.135149 (+ 118us) negotiation.cc:107] Waiting for socket to connect
0811 02:52:33.135166 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 02:52:33.135245 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:52:33.135456 (+ 211us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:52:33.135458 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:52:33.135593 (+ 135us) client_negotiation.cc:189] Negotiated authn=SASL
0811 02:52:33.135694 (+ 101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:52:33.135699 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:52:33.136492 (+ 793us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:52:33.136494 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:52:33.137285 (+ 791us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:52:33.137295 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:52:33.137805 (+ 510us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:52:33.137808 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:52:33.137896 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:52:33.138325 (+ 429us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 02:52:33.138337 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0811 02:52:33.140379 (+ 2042us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.12.138.254:34683: Server oryx/127.12.138.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":47}
W20260811 02:52:33.140666 23471 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 02:52:33.135115 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.138.195:43899 (local address 127.12.138.254:34683)
0811 02:52:33.135235 (+ 120us) server_negotiation.cc:206] Beginning negotiation
0811 02:52:33.135238 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 02:52:33.135250 (+ 12us) server_negotiation.cc:407] Connection header received
0811 02:52:33.135329 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:52:33.135331 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:52:33.135375 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:52:33.135443 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:52:33.135880 (+ 437us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:52:33.136391 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:52:33.137394 (+ 1003us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:52:33.137603 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:52:33.137685 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:52:33.140515 (+ 2830us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.12.138.195:43899: BlockingRecv error: recv got EOF from 127.12.138.195:43899 (error 108)
Metrics: {"server-negotiator.queue_time_us":48}
Aug 11 02:52:33 dist-test-slave-9gcw krb5kdc[22473](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.138.195@KRBTEST.COM for oryx/127.12.138.254@KRBTEST.COM, Server not found in Kerberos database
Aug 11 02:52:33 dist-test-slave-9gcw krb5kdc[22473](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.138.195@KRBTEST.COM for oryx/127.12.138.254@KRBTEST.COM, Server not found in Kerberos database
I20260811 02:52:33.146237 23624 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:52:33.141085 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.138.254:34683 (local address 127.12.138.195:34387)
0811 02:52:33.141214 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0811 02:52:33.141230 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 02:52:33.141336 (+ 106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:52:33.141604 (+ 268us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:52:33.141607 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:52:33.141764 (+ 157us) client_negotiation.cc:189] Negotiated authn=SASL
0811 02:52:33.141849 (+ 85us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:52:33.141854 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:52:33.142630 (+ 776us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:52:33.142634 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:52:33.143272 (+ 638us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:52:33.143282 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:52:33.143725 (+ 443us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:52:33.143728 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:52:33.143826 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:52:33.144313 (+ 487us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 02:52:33.144327 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0811 02:52:33.146098 (+ 1771us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.12.138.254:34683: Server oryx/127.12.138.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":48}
W20260811 02:52:33.146333 23471 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 02:52:33.141219 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.138.195:34387 (local address 127.12.138.254:34683)
0811 02:52:33.141409 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0811 02:52:33.141415 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 02:52:33.141435 (+ 20us) server_negotiation.cc:407] Connection header received
0811 02:52:33.141465 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:52:33.141466 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:52:33.141501 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:52:33.141587 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:52:33.141975 (+ 388us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:52:33.142498 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:52:33.143392 (+ 894us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:52:33.143606 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:52:33.143718 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:52:33.146209 (+ 2491us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.12.138.195:34387: BlockingRecv error: recv got EOF from 127.12.138.195:34387 (error 108)
Metrics: {"server-negotiator.queue_time_us":100}
W20260811 02:52:33.146548 23621 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
Aug 11 02:52:33 dist-test-slave-9gcw krb5kdc[22473](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.138.193@KRBTEST.COM for oryx/127.12.138.254@KRBTEST.COM, Server not found in Kerberos database
Aug 11 02:52:33 dist-test-slave-9gcw krb5kdc[22473](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.138.193@KRBTEST.COM for oryx/127.12.138.254@KRBTEST.COM, Server not found in Kerberos database
I20260811 02:52:33.734733 23625 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:52:33.728880 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.138.254:34683 (local address 127.12.138.193:38007)
0811 02:52:33.729294 (+ 414us) negotiation.cc:107] Waiting for socket to connect
0811 02:52:33.729307 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 02:52:33.729413 (+ 106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:52:33.729676 (+ 263us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:52:33.729678 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:52:33.729882 (+ 204us) client_negotiation.cc:189] Negotiated authn=SASL
0811 02:52:33.730008 (+ 126us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:52:33.730014 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:52:33.730749 (+ 735us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:52:33.730752 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:52:33.731327 (+ 575us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:52:33.731335 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:52:33.731810 (+ 475us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:52:33.731814 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:52:33.731921 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:52:33.732530 (+ 609us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 02:52:33.732548 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0811 02:52:33.734615 (+ 2067us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.12.138.254:34683: Server oryx/127.12.138.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":328,"thread_start_us":201,"threads_started":1}
W20260811 02:52:33.734915 23626 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 02:52:33.729001 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.138.193:38007 (local address 127.12.138.254:34683)
0811 02:52:33.729281 (+ 280us) server_negotiation.cc:206] Beginning negotiation
0811 02:52:33.729286 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 02:52:33.729379 (+ 93us) server_negotiation.cc:407] Connection header received
0811 02:52:33.729499 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:52:33.729501 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:52:33.729546 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:52:33.729632 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:52:33.730117 (+ 485us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:52:33.730655 (+ 538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:52:33.731428 (+ 773us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:52:33.731644 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:52:33.731782 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:52:33.734747 (+ 2965us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.12.138.193:38007: BlockingRecv error: recv got EOF from 127.12.138.193:38007 (error 108)
Metrics: {"server-negotiator.queue_time_us":196,"thread_start_us":117,"threads_started":1}
Aug 11 02:52:33 dist-test-slave-9gcw krb5kdc[22473](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.138.194@KRBTEST.COM for oryx/127.12.138.254@KRBTEST.COM, Server not found in Kerberos database
Aug 11 02:52:33 dist-test-slave-9gcw krb5kdc[22473](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.138.194@KRBTEST.COM for oryx/127.12.138.254@KRBTEST.COM, Server not found in Kerberos database
I20260811 02:52:33.947553 23627 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:52:33.941118 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.138.254:34683 (local address 127.12.138.194:45667)
0811 02:52:33.941531 (+ 413us) negotiation.cc:107] Waiting for socket to connect
0811 02:52:33.941545 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 02:52:33.941780 (+ 235us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:52:33.942060 (+ 280us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:52:33.942064 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:52:33.942294 (+ 230us) client_negotiation.cc:189] Negotiated authn=SASL
0811 02:52:33.942391 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:52:33.942397 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:52:33.943342 (+ 945us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:52:33.943348 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:52:33.944018 (+ 670us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:52:33.944027 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:52:33.944482 (+ 455us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:52:33.944485 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:52:33.944563 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:52:33.945080 (+ 517us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 02:52:33.945092 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0811 02:52:33.947426 (+ 2334us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.12.138.254:34683: Server oryx/127.12.138.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":282,"thread_start_us":117,"threads_started":1}
W20260811 02:52:33.947862 23626 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 02:52:33.941238 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.138.194:45667 (local address 127.12.138.254:34683)
0811 02:52:33.941391 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0811 02:52:33.941395 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 02:52:33.941668 (+ 273us) server_negotiation.cc:407] Connection header received
0811 02:52:33.941889 (+ 221us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:52:33.941892 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:52:33.941947 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:52:33.942042 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:52:33.942512 (+ 470us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:52:33.943227 (+ 715us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:52:33.944146 (+ 919us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:52:33.944371 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:52:33.944481 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:52:33.947720 (+ 3239us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.12.138.194:45667: BlockingRecv error: recv got EOF from 127.12.138.194:45667 (error 108)
Metrics: {"server-negotiator.queue_time_us":61}
Aug 11 02:52:34 dist-test-slave-9gcw krb5kdc[22473](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.138.195@KRBTEST.COM for oryx/127.12.138.254@KRBTEST.COM, Server not found in Kerberos database
Aug 11 02:52:34 dist-test-slave-9gcw krb5kdc[22473](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.138.195@KRBTEST.COM for oryx/127.12.138.254@KRBTEST.COM, Server not found in Kerberos database
I20260811 02:52:34.152943 23629 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:52:34.147246 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.138.254:34683 (local address 127.12.138.195:57363)
0811 02:52:34.147561 (+ 315us) negotiation.cc:107] Waiting for socket to connect
0811 02:52:34.147575 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 02:52:34.147682 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:52:34.147945 (+ 263us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:52:34.147947 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:52:34.148134 (+ 187us) client_negotiation.cc:189] Negotiated authn=SASL
0811 02:52:34.148228 (+ 94us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:52:34.148233 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:52:34.149002 (+ 769us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:52:34.149008 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:52:34.149743 (+ 735us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:52:34.149754 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:52:34.150240 (+ 486us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:52:34.150244 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:52:34.150367 (+ 123us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:52:34.150953 (+ 586us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 02:52:34.150965 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0811 02:52:34.152820 (+ 1855us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.12.138.254:34683: Server oryx/127.12.138.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":234,"thread_start_us":109,"threads_started":1}
W20260811 02:52:34.153124 23626 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 02:52:34.147306 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.138.195:57363 (local address 127.12.138.254:34683)
0811 02:52:34.147465 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0811 02:52:34.147470 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 02:52:34.147656 (+ 186us) server_negotiation.cc:407] Connection header received
0811 02:52:34.147770 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:52:34.147772 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:52:34.147835 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:52:34.147906 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:52:34.148335 (+ 429us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:52:34.148872 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:52:34.149877 (+ 1005us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:52:34.150113 (+ 236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:52:34.150224 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:52:34.152963 (+ 2739us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.12.138.195:57363: BlockingRecv error: recv got EOF from 127.12.138.195:57363 (error 108)
Metrics: {"server-negotiator.queue_time_us":62}
Aug 11 02:52:34 dist-test-slave-9gcw krb5kdc[22473](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.138.193@KRBTEST.COM for oryx/127.12.138.254@KRBTEST.COM, Server not found in Kerberos database
Aug 11 02:52:34 dist-test-slave-9gcw krb5kdc[22473](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.138.193@KRBTEST.COM for oryx/127.12.138.254@KRBTEST.COM, Server not found in Kerberos database
I20260811 02:52:34.740967 23630 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:52:34.735594 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.138.254:34683 (local address 127.12.138.193:55861)
0811 02:52:34.735914 (+ 320us) negotiation.cc:107] Waiting for socket to connect
0811 02:52:34.735932 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 02:52:34.736052 (+ 120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:52:34.736324 (+ 272us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:52:34.736326 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:52:34.736521 (+ 195us) client_negotiation.cc:189] Negotiated authn=SASL
0811 02:52:34.736622 (+ 101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:52:34.736627 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:52:34.737428 (+ 801us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:52:34.737431 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:52:34.737988 (+ 557us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:52:34.737996 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:52:34.738419 (+ 423us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:52:34.738421 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:52:34.738486 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:52:34.738986 (+ 500us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 02:52:34.738997 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0811 02:52:34.740849 (+ 1852us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.12.138.254:34683: Server oryx/127.12.138.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":230,"thread_start_us":88,"threads_started":1}
W20260811 02:52:34.741127 23631 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 02:52:34.735646 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.138.193:55861 (local address 127.12.138.254:34683)
0811 02:52:34.735887 (+ 241us) server_negotiation.cc:206] Beginning negotiation
0811 02:52:34.735893 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 02:52:34.736014 (+ 121us) server_negotiation.cc:407] Connection header received
0811 02:52:34.736134 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:52:34.736136 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:52:34.736182 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:52:34.736268 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:52:34.736727 (+ 459us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:52:34.737332 (+ 605us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:52:34.738088 (+ 756us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:52:34.738321 (+ 233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:52:34.738394 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:52:34.740992 (+ 2598us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.12.138.193:55861: BlockingRecv error: recv got EOF from 127.12.138.193:55861 (error 108)
Metrics: {"server-negotiator.queue_time_us":152,"thread_start_us":65,"threads_started":1}
Aug 11 02:52:34 dist-test-slave-9gcw krb5kdc[22473](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.138.194@KRBTEST.COM for oryx/127.12.138.254@KRBTEST.COM, Server not found in Kerberos database
Aug 11 02:52:34 dist-test-slave-9gcw krb5kdc[22473](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.138.194@KRBTEST.COM for oryx/127.12.138.254@KRBTEST.COM, Server not found in Kerberos database
I20260811 02:52:34.959406 23632 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:52:34.948569 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.138.254:34683 (local address 127.12.138.194:51437)
0811 02:52:34.948902 (+ 333us) negotiation.cc:107] Waiting for socket to connect
0811 02:52:34.948920 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 02:52:34.949034 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:52:34.949365 (+ 331us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:52:34.949368 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:52:34.949594 (+ 226us) client_negotiation.cc:189] Negotiated authn=SASL
0811 02:52:34.949692 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:52:34.949697 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:52:34.950486 (+ 789us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:52:34.950491 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:52:34.951069 (+ 578us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:52:34.951077 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:52:34.951555 (+ 478us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:52:34.951558 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:52:34.951647 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:52:34.957105 (+ 5458us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 02:52:34.957120 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0811 02:52:34.959240 (+ 2120us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.12.138.254:34683: Server oryx/127.12.138.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":243,"thread_start_us":109,"threads_started":1}
W20260811 02:52:34.959568 23631 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 02:52:34.948707 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.138.194:51437 (local address 127.12.138.254:34683)
0811 02:52:34.948902 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0811 02:52:34.948908 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 02:52:34.948981 (+ 73us) server_negotiation.cc:407] Connection header received
0811 02:52:34.949171 (+ 190us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:52:34.949174 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:52:34.949231 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:52:34.949346 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:52:34.949823 (+ 477us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:52:34.950369 (+ 546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:52:34.951198 (+ 829us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:52:34.951435 (+ 237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:52:34.951549 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:52:34.959437 (+ 7888us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.12.138.194:51437: BlockingRecv error: recv got EOF from 127.12.138.194:51437 (error 108)
Metrics: {"server-negotiator.queue_time_us":67}
Aug 11 02:52:35 dist-test-slave-9gcw krb5kdc[22473](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.138.195@KRBTEST.COM for oryx/127.12.138.254@KRBTEST.COM, Server not found in Kerberos database
Aug 11 02:52:35 dist-test-slave-9gcw krb5kdc[22473](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.138.195@KRBTEST.COM for oryx/127.12.138.254@KRBTEST.COM, Server not found in Kerberos database
I20260811 02:52:35.159502 23633 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:52:35.153818 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.138.254:34683 (local address 127.12.138.195:44021)
0811 02:52:35.154136 (+ 318us) negotiation.cc:107] Waiting for socket to connect
0811 02:52:35.154153 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 02:52:35.154274 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:52:35.154565 (+ 291us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:52:35.154568 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:52:35.154781 (+ 213us) client_negotiation.cc:189] Negotiated authn=SASL
0811 02:52:35.154882 (+ 101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:52:35.154888 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:52:35.155679 (+ 791us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:52:35.155683 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:52:35.156250 (+ 567us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:52:35.156259 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:52:35.156741 (+ 482us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:52:35.156744 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:52:35.156838 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:52:35.157487 (+ 649us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 02:52:35.157499 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0811 02:52:35.159308 (+ 1809us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.12.138.254:34683: Server oryx/127.12.138.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":222,"thread_start_us":105,"threads_started":1}
W20260811 02:52:35.159598 23631 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 02:52:35.153998 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.138.195:44021 (local address 127.12.138.254:34683)
0811 02:52:35.154136 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0811 02:52:35.154142 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 02:52:35.154215 (+ 73us) server_negotiation.cc:407] Connection header received
0811 02:52:35.154389 (+ 174us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:52:35.154393 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:52:35.154448 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:52:35.154537 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:52:35.155001 (+ 464us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:52:35.155563 (+ 562us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:52:35.156380 (+ 817us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:52:35.156623 (+ 243us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:52:35.156737 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:52:35.159443 (+ 2706us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.12.138.195:44021: BlockingRecv error: recv got EOF from 127.12.138.195:44021 (error 108)
Metrics: {"server-negotiator.queue_time_us":52}
Aug 11 02:52:35 dist-test-slave-9gcw krb5kdc[22473](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.138.193@KRBTEST.COM for oryx/127.12.138.254@KRBTEST.COM, Server not found in Kerberos database
Aug 11 02:52:35 dist-test-slave-9gcw krb5kdc[22473](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.138.193@KRBTEST.COM for oryx/127.12.138.254@KRBTEST.COM, Server not found in Kerberos database
I20260811 02:52:35.747599 23634 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:52:35.741920 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.138.254:34683 (local address 127.12.138.193:54611)
0811 02:52:35.742215 (+ 295us) negotiation.cc:107] Waiting for socket to connect
0811 02:52:35.742228 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 02:52:35.742344 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:52:35.742586 (+ 242us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:52:35.742588 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:52:35.742775 (+ 187us) client_negotiation.cc:189] Negotiated authn=SASL
0811 02:52:35.742949 (+ 174us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:52:35.742956 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:52:35.743732 (+ 776us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:52:35.743735 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:52:35.744374 (+ 639us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:52:35.744384 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:52:35.744836 (+ 452us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:52:35.744840 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:52:35.744912 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:52:35.745531 (+ 619us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 02:52:35.745544 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0811 02:52:35.747470 (+ 1926us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.12.138.254:34683: Server oryx/127.12.138.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":198,"thread_start_us":108,"threads_started":1}
W20260811 02:52:35.747754 23635 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 02:52:35.741961 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.138.193:54611 (local address 127.12.138.254:34683)
0811 02:52:35.742216 (+ 255us) server_negotiation.cc:206] Beginning negotiation
0811 02:52:35.742220 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 02:52:35.742301 (+ 81us) server_negotiation.cc:407] Connection header received
0811 02:52:35.742424 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:52:35.742426 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:52:35.742472 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:52:35.742551 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:52:35.743038 (+ 487us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:52:35.743596 (+ 558us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:52:35.744490 (+ 894us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:52:35.744719 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:52:35.744809 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:52:35.747616 (+ 2807us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.12.138.193:54611: BlockingRecv error: recv got EOF from 127.12.138.193:54611 (error 108)
Metrics: {"server-negotiator.queue_time_us":179,"thread_start_us":84,"threads_started":1}
Aug 11 02:52:35 dist-test-slave-9gcw krb5kdc[22473](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.138.194@KRBTEST.COM for oryx/127.12.138.254@KRBTEST.COM, Server not found in Kerberos database
Aug 11 02:52:35 dist-test-slave-9gcw krb5kdc[22473](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.138.194@KRBTEST.COM for oryx/127.12.138.254@KRBTEST.COM, Server not found in Kerberos database
I20260811 02:52:35.966144 23636 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:52:35.960270 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.138.254:34683 (local address 127.12.138.194:35333)
0811 02:52:35.960636 (+ 366us) negotiation.cc:107] Waiting for socket to connect
0811 02:52:35.960652 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 02:52:35.960800 (+ 148us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:52:35.961093 (+ 293us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:52:35.961096 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:52:35.961369 (+ 273us) client_negotiation.cc:189] Negotiated authn=SASL
0811 02:52:35.961472 (+ 103us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:52:35.961478 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:52:35.962264 (+ 786us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:52:35.962269 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:52:35.962848 (+ 579us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:52:35.962856 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:52:35.963327 (+ 471us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:52:35.963331 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:52:35.963420 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:52:35.963957 (+ 537us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 02:52:35.963968 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0811 02:52:35.965984 (+ 2016us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.12.138.254:34683: Server oryx/127.12.138.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":281,"thread_start_us":100,"threads_started":1}
W20260811 02:52:35.966305 23635 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 02:52:35.960413 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.138.194:35333 (local address 127.12.138.254:34683)
0811 02:52:35.960585 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0811 02:52:35.960590 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 02:52:35.960742 (+ 152us) server_negotiation.cc:407] Connection header received
0811 02:52:35.960920 (+ 178us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:52:35.960923 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:52:35.960977 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:52:35.961068 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:52:35.961600 (+ 532us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:52:35.962144 (+ 544us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:52:35.962975 (+ 831us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:52:35.963208 (+ 233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:52:35.963320 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:52:35.966172 (+ 2852us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.12.138.194:35333: BlockingRecv error: recv got EOF from 127.12.138.194:35333 (error 108)
Metrics: {"server-negotiator.queue_time_us":63}
Aug 11 02:52:36 dist-test-slave-9gcw krb5kdc[22473](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.138.195@KRBTEST.COM for oryx/127.12.138.254@KRBTEST.COM, Server not found in Kerberos database
Aug 11 02:52:36 dist-test-slave-9gcw krb5kdc[22473](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.138.195@KRBTEST.COM for oryx/127.12.138.254@KRBTEST.COM, Server not found in Kerberos database
I20260811 02:52:36.165997 23637 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:52:36.160506 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.138.254:34683 (local address 127.12.138.195:58579)
0811 02:52:36.160863 (+ 357us) negotiation.cc:107] Waiting for socket to connect
0811 02:52:36.160877 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 02:52:36.160978 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:52:36.161237 (+ 259us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:52:36.161239 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:52:36.161429 (+ 190us) client_negotiation.cc:189] Negotiated authn=SASL
0811 02:52:36.161557 (+ 128us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:52:36.161566 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:52:36.162321 (+ 755us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:52:36.162323 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:52:36.162877 (+ 554us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:52:36.162885 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:52:36.163287 (+ 402us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:52:36.163289 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:52:36.163352 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:52:36.163916 (+ 564us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 02:52:36.163927 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0811 02:52:36.165832 (+ 1905us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.12.138.254:34683: Server oryx/127.12.138.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":282,"thread_start_us":100,"threads_started":1}
W20260811 02:52:36.166114 23635 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 02:52:36.160567 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.138.195:58579 (local address 127.12.138.254:34683)
0811 02:52:36.160735 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0811 02:52:36.160740 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 02:52:36.160952 (+ 212us) server_negotiation.cc:407] Connection header received
0811 02:52:36.161060 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:52:36.161062 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:52:36.161111 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:52:36.161202 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:52:36.161675 (+ 473us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:52:36.162220 (+ 545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:52:36.162981 (+ 761us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:52:36.163193 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:52:36.163269 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:52:36.165974 (+ 2705us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.12.138.195:58579: BlockingRecv error: recv got EOF from 127.12.138.195:58579 (error 108)
Metrics: {"server-negotiator.queue_time_us":76}
Aug 11 02:52:36 dist-test-slave-9gcw krb5kdc[22473](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.138.193@KRBTEST.COM for oryx/127.12.138.254@KRBTEST.COM, Server not found in Kerberos database
Aug 11 02:52:36 dist-test-slave-9gcw krb5kdc[22473](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.138.193@KRBTEST.COM for oryx/127.12.138.254@KRBTEST.COM, Server not found in Kerberos database
I20260811 02:52:36.754733 23638 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:52:36.748687 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.138.254:34683 (local address 127.12.138.193:38967)
0811 02:52:36.749025 (+ 338us) negotiation.cc:107] Waiting for socket to connect
0811 02:52:36.749039 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 02:52:36.749131 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:52:36.749492 (+ 361us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:52:36.749494 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:52:36.749683 (+ 189us) client_negotiation.cc:189] Negotiated authn=SASL
0811 02:52:36.749809 (+ 126us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:52:36.749815 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:52:36.750628 (+ 813us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:52:36.750631 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:52:36.751227 (+ 596us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:52:36.751237 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:52:36.751918 (+ 681us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:52:36.751921 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:52:36.751985 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:52:36.752607 (+ 622us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 02:52:36.752624 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 02:52:36.754605 (+ 1981us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.12.138.254:34683: Server oryx/127.12.138.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":233,"thread_start_us":148,"threads_started":1}
W20260811 02:52:36.754972 23639 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 02:52:36.748771 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.138.193:38967 (local address 127.12.138.254:34683)
0811 02:52:36.749253 (+ 482us) server_negotiation.cc:206] Beginning negotiation
0811 02:52:36.749257 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 02:52:36.749273 (+ 16us) server_negotiation.cc:407] Connection header received
0811 02:52:36.749316 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:52:36.749318 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:52:36.749361 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:52:36.749457 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:52:36.749940 (+ 483us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:52:36.750506 (+ 566us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:52:36.751451 (+ 945us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:52:36.751798 (+ 347us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:52:36.751903 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:52:36.754811 (+ 2908us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.12.138.193:38967: BlockingRecv error: recv got EOF from 127.12.138.193:38967 (error 108)
Metrics: {"server-negotiator.queue_time_us":352,"thread_start_us":258,"threads_started":1}
Aug 11 02:52:36 dist-test-slave-9gcw krb5kdc[22473](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.138.194@KRBTEST.COM for oryx/127.12.138.254@KRBTEST.COM, Server not found in Kerberos database
Aug 11 02:52:36 dist-test-slave-9gcw krb5kdc[22473](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.138.194@KRBTEST.COM for oryx/127.12.138.254@KRBTEST.COM, Server not found in Kerberos database
I20260811 02:52:36.973322 23640 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:52:36.967005 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.138.254:34683 (local address 127.12.138.194:33985)
0811 02:52:36.967339 (+ 334us) negotiation.cc:107] Waiting for socket to connect
0811 02:52:36.967362 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0811 02:52:36.967518 (+ 156us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:52:36.967804 (+ 286us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:52:36.967807 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:52:36.968048 (+ 241us) client_negotiation.cc:189] Negotiated authn=SASL
0811 02:52:36.968220 (+ 172us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:52:36.968232 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:52:36.969361 (+ 1129us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:52:36.969366 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:52:36.969995 (+ 629us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:52:36.970007 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:52:36.970501 (+ 494us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:52:36.970504 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:52:36.970572 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:52:36.971250 (+ 678us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 02:52:36.971262 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0811 02:52:36.973135 (+ 1873us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.12.138.254:34683: Server oryx/127.12.138.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":241,"thread_start_us":90,"threads_started":1}
W20260811 02:52:36.973520 23639 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 02:52:36.967108 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.138.194:33985 (local address 127.12.138.254:34683)
0811 02:52:36.967262 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0811 02:52:36.967268 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 02:52:36.967474 (+ 206us) server_negotiation.cc:407] Connection header received
0811 02:52:36.967638 (+ 164us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:52:36.967641 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:52:36.967689 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:52:36.967754 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:52:36.968372 (+ 618us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:52:36.969174 (+ 802us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:52:36.970127 (+ 953us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:52:36.970359 (+ 232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:52:36.970483 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:52:36.973383 (+ 2900us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.12.138.194:33985: BlockingRecv error: recv got EOF from 127.12.138.194:33985 (error 108)
Metrics: {"server-negotiator.queue_time_us":60}
Aug 11 02:52:37 dist-test-slave-9gcw krb5kdc[22473](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.138.195@KRBTEST.COM for oryx/127.12.138.254@KRBTEST.COM, Server not found in Kerberos database
Aug 11 02:52:37 dist-test-slave-9gcw krb5kdc[22473](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.138.195@KRBTEST.COM for oryx/127.12.138.254@KRBTEST.COM, Server not found in Kerberos database
I20260811 02:52:37.173152 23641 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:52:37.166912 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.138.254:34683 (local address 127.12.138.195:53561)
0811 02:52:37.167228 (+ 316us) negotiation.cc:107] Waiting for socket to connect
0811 02:52:37.167251 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0811 02:52:37.167408 (+ 157us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:52:37.167700 (+ 292us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:52:37.167704 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:52:37.167934 (+ 230us) client_negotiation.cc:189] Negotiated authn=SASL
0811 02:52:37.168031 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:52:37.168037 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:52:37.168894 (+ 857us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:52:37.168899 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:52:37.169512 (+ 613us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:52:37.169521 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:52:37.170034 (+ 513us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:52:37.170037 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:52:37.170105 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:52:37.170709 (+ 604us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 02:52:37.170723 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0811 02:52:37.172982 (+ 2259us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.12.138.254:34683: Server oryx/127.12.138.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":232,"thread_start_us":90,"threads_started":1}
W20260811 02:52:37.173454 23639 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 02:52:37.166992 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.138.195:53561 (local address 127.12.138.254:34683)
0811 02:52:37.167184 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0811 02:52:37.167189 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 02:52:37.167342 (+ 153us) server_negotiation.cc:407] Connection header received
0811 02:52:37.167521 (+ 179us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:52:37.167526 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:52:37.167581 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:52:37.167677 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:52:37.168163 (+ 486us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:52:37.168747 (+ 584us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:52:37.169670 (+ 923us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:52:37.169922 (+ 252us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:52:37.170013 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:52:37.173126 (+ 3113us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.12.138.195:53561: BlockingRecv error: recv got EOF from 127.12.138.195:53561 (error 108)
Metrics: {"server-negotiator.queue_time_us":84}
Aug 11 02:52:37 dist-test-slave-9gcw krb5kdc[22473](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.138.193@KRBTEST.COM for oryx/127.12.138.254@KRBTEST.COM, Server not found in Kerberos database
Aug 11 02:52:37 dist-test-slave-9gcw krb5kdc[22473](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.138.193@KRBTEST.COM for oryx/127.12.138.254@KRBTEST.COM, Server not found in Kerberos database
I20260811 02:52:37.762035 23642 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:52:37.755543 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.138.254:34683 (local address 127.12.138.193:38697)
0811 02:52:37.755906 (+ 363us) negotiation.cc:107] Waiting for socket to connect
0811 02:52:37.755922 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 02:52:37.756043 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:52:37.756334 (+ 291us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:52:37.756336 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:52:37.756531 (+ 195us) client_negotiation.cc:189] Negotiated authn=SASL
0811 02:52:37.756704 (+ 173us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:52:37.756712 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:52:37.757518 (+ 806us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:52:37.757522 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:52:37.758249 (+ 727us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:52:37.758260 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:52:37.758763 (+ 503us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:52:37.758766 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:52:37.758887 (+ 121us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:52:37.759474 (+ 587us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 02:52:37.759487 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0811 02:52:37.761876 (+ 2389us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.12.138.254:34683: Server oryx/127.12.138.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":275,"thread_start_us":96,"threads_started":1}
W20260811 02:52:37.762200 23643 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 02:52:37.755684 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.138.193:38697 (local address 127.12.138.254:34683)
0811 02:52:37.755940 (+ 256us) server_negotiation.cc:206] Beginning negotiation
0811 02:52:37.755943 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 02:52:37.755986 (+ 43us) server_negotiation.cc:407] Connection header received
0811 02:52:37.756139 (+ 153us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:52:37.756141 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:52:37.756185 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:52:37.756264 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:52:37.756821 (+ 557us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:52:37.757382 (+ 561us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:52:37.758373 (+ 991us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:52:37.758612 (+ 239us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:52:37.758695 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:52:37.762041 (+ 3346us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.12.138.193:38697: BlockingRecv error: recv got EOF from 127.12.138.193:38697 (error 108)
Metrics: {"server-negotiator.queue_time_us":164,"thread_start_us":62,"threads_started":1}
Aug 11 02:52:37 dist-test-slave-9gcw krb5kdc[22473](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.138.194@KRBTEST.COM for oryx/127.12.138.254@KRBTEST.COM, Server not found in Kerberos database
Aug 11 02:52:37 dist-test-slave-9gcw krb5kdc[22473](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.138.194@KRBTEST.COM for oryx/127.12.138.254@KRBTEST.COM, Server not found in Kerberos database
I20260811 02:52:37.980747 23644 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:52:37.974178 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.138.254:34683 (local address 127.12.138.194:52425)
0811 02:52:37.974533 (+ 355us) negotiation.cc:107] Waiting for socket to connect
0811 02:52:37.974552 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0811 02:52:37.974769 (+ 217us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:52:37.975097 (+ 328us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:52:37.975105 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:52:37.975359 (+ 254us) client_negotiation.cc:189] Negotiated authn=SASL
0811 02:52:37.975458 (+ 99us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:52:37.975464 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:52:37.976445 (+ 981us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:52:37.976452 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:52:37.977068 (+ 616us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:52:37.977079 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:52:37.977650 (+ 571us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:52:37.977654 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:52:37.977742 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:52:37.978270 (+ 528us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 02:52:37.978282 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0811 02:52:37.980565 (+ 2283us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.12.138.254:34683: Server oryx/127.12.138.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":238,"thread_start_us":122,"threads_started":1}
W20260811 02:52:37.980814 23643 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 02:52:37.974378 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.138.194:52425 (local address 127.12.138.254:34683)
0811 02:52:37.974614 (+ 236us) server_negotiation.cc:206] Beginning negotiation
0811 02:52:37.974618 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 02:52:37.974633 (+ 15us) server_negotiation.cc:407] Connection header received
0811 02:52:37.974892 (+ 259us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:52:37.974895 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:52:37.974950 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:52:37.975054 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:52:37.975593 (+ 539us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:52:37.976304 (+ 711us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:52:37.977278 (+ 974us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:52:37.977525 (+ 247us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:52:37.977651 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:52:37.980701 (+ 3050us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.12.138.194:52425: BlockingRecv error: recv got EOF from 127.12.138.194:52425 (error 108)
Metrics: {"server-negotiator.queue_time_us":111}
Aug 11 02:52:38 dist-test-slave-9gcw krb5kdc[22473](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.138.195@KRBTEST.COM for oryx/127.12.138.254@KRBTEST.COM, Server not found in Kerberos database
Aug 11 02:52:38 dist-test-slave-9gcw krb5kdc[22473](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.138.195@KRBTEST.COM for oryx/127.12.138.254@KRBTEST.COM, Server not found in Kerberos database
I20260811 02:52:38.179633 23645 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:52:38.173996 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.138.254:34683 (local address 127.12.138.195:58223)
0811 02:52:38.174306 (+ 310us) negotiation.cc:107] Waiting for socket to connect
0811 02:52:38.174323 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 02:52:38.174489 (+ 166us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:52:38.174772 (+ 283us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:52:38.174774 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:52:38.174982 (+ 208us) client_negotiation.cc:189] Negotiated authn=SASL
0811 02:52:38.175081 (+ 99us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:52:38.175086 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:52:38.175914 (+ 828us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:52:38.175916 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:52:38.176476 (+ 560us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:52:38.176484 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:52:38.176922 (+ 438us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:52:38.176924 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:52:38.176985 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:52:38.177538 (+ 553us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 02:52:38.177550 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0811 02:52:38.179456 (+ 1906us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.12.138.254:34683: Server oryx/127.12.138.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":206,"thread_start_us":88,"threads_started":1}
W20260811 02:52:38.179695 23643 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 02:52:38.174074 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.138.195:58223 (local address 127.12.138.254:34683)
0811 02:52:38.174249 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0811 02:52:38.174255 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 02:52:38.174410 (+ 155us) server_negotiation.cc:407] Connection header received
0811 02:52:38.174613 (+ 203us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:52:38.174616 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:52:38.174666 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:52:38.174738 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:52:38.175277 (+ 539us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:52:38.175816 (+ 539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:52:38.176591 (+ 775us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:52:38.176821 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:52:38.176904 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:52:38.179583 (+ 2679us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.12.138.195:58223: BlockingRecv error: recv got EOF from 127.12.138.195:58223 (error 108)
Metrics: {"server-negotiator.queue_time_us":75}
Aug 11 02:52:38 dist-test-slave-9gcw krb5kdc[22473](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.138.193@KRBTEST.COM for oryx/127.12.138.254@KRBTEST.COM, Server not found in Kerberos database
Aug 11 02:52:38 dist-test-slave-9gcw krb5kdc[22473](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.138.193@KRBTEST.COM for oryx/127.12.138.254@KRBTEST.COM, Server not found in Kerberos database
I20260811 02:52:38.769306 23646 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:52:38.763010 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.138.254:34683 (local address 127.12.138.193:34769)
0811 02:52:38.763390 (+ 380us) negotiation.cc:107] Waiting for socket to connect
0811 02:52:38.763411 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0811 02:52:38.763541 (+ 130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:52:38.763915 (+ 374us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:52:38.763920 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:52:38.764191 (+ 271us) client_negotiation.cc:189] Negotiated authn=SASL
0811 02:52:38.764363 (+ 172us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:52:38.764373 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:52:38.765338 (+ 965us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:52:38.765341 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:52:38.765892 (+ 551us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:52:38.765900 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:52:38.766358 (+ 458us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:52:38.766361 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:52:38.766427 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:52:38.766956 (+ 529us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 02:52:38.766967 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0811 02:52:38.769041 (+ 2074us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.12.138.254:34683: Server oryx/127.12.138.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":254,"thread_start_us":137,"threads_started":1}
W20260811 02:52:38.769409 23647 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 02:52:38.763077 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.138.193:34769 (local address 127.12.138.254:34683)
0811 02:52:38.763413 (+ 336us) server_negotiation.cc:206] Beginning negotiation
0811 02:52:38.763421 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 02:52:38.763504 (+ 83us) server_negotiation.cc:407] Connection header received
0811 02:52:38.763674 (+ 170us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:52:38.763678 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:52:38.763746 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:52:38.763861 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:52:38.764485 (+ 624us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:52:38.765210 (+ 725us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:52:38.765998 (+ 788us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:52:38.766235 (+ 237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:52:38.766331 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:52:38.769249 (+ 2918us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.12.138.193:34769: BlockingRecv error: recv got EOF from 127.12.138.193:34769 (error 108)
Metrics: {"server-negotiator.queue_time_us":213,"thread_start_us":87,"threads_started":1}
Aug 11 02:52:38 dist-test-slave-9gcw krb5kdc[22473](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.138.194@KRBTEST.COM for oryx/127.12.138.254@KRBTEST.COM, Server not found in Kerberos database
Aug 11 02:52:38 dist-test-slave-9gcw krb5kdc[22473](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.138.194@KRBTEST.COM for oryx/127.12.138.254@KRBTEST.COM, Server not found in Kerberos database
I20260811 02:52:38.987672 23648 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:52:38.981744 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.138.254:34683 (local address 127.12.138.194:48033)
0811 02:52:38.982058 (+ 314us) negotiation.cc:107] Waiting for socket to connect
0811 02:52:38.982073 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 02:52:38.982183 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:52:38.982442 (+ 259us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:52:38.982444 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:52:38.982638 (+ 194us) client_negotiation.cc:189] Negotiated authn=SASL
0811 02:52:38.982734 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:52:38.982739 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:52:38.983552 (+ 813us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:52:38.983555 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:52:38.984241 (+ 686us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:52:38.984259 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:52:38.984748 (+ 489us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:52:38.984751 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:52:38.984831 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:52:38.985455 (+ 624us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 02:52:38.985467 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0811 02:52:38.987537 (+ 2070us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.12.138.254:34683: Server oryx/127.12.138.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":217,"thread_start_us":98,"threads_started":1}
W20260811 02:52:38.987888 23647 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 02:52:38.981868 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.138.194:48033 (local address 127.12.138.254:34683)
0811 02:52:38.982041 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0811 02:52:38.982045 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 02:52:38.982154 (+ 109us) server_negotiation.cc:407] Connection header received
0811 02:52:38.982271 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:52:38.982274 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:52:38.982321 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:52:38.982405 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:52:38.982854 (+ 449us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:52:38.983445 (+ 591us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:52:38.984385 (+ 940us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:52:38.984626 (+ 241us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:52:38.984744 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:52:38.987696 (+ 2952us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.12.138.194:48033: BlockingRecv error: recv got EOF from 127.12.138.194:48033 (error 108)
Metrics: {"server-negotiator.queue_time_us":71}
Aug 11 02:52:39 dist-test-slave-9gcw krb5kdc[22473](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.138.195@KRBTEST.COM for oryx/127.12.138.254@KRBTEST.COM, Server not found in Kerberos database
Aug 11 02:52:39 dist-test-slave-9gcw krb5kdc[22473](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.138.195@KRBTEST.COM for oryx/127.12.138.254@KRBTEST.COM, Server not found in Kerberos database
I20260811 02:52:39.186424 23649 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:52:39.180494 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.138.254:34683 (local address 127.12.138.195:41399)
0811 02:52:39.180872 (+ 378us) negotiation.cc:107] Waiting for socket to connect
0811 02:52:39.180884 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 02:52:39.181022 (+ 138us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:52:39.181342 (+ 320us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:52:39.181345 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:52:39.181560 (+ 215us) client_negotiation.cc:189] Negotiated authn=SASL
0811 02:52:39.181656 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:52:39.181662 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:52:39.182441 (+ 779us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:52:39.182447 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:52:39.183003 (+ 556us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:52:39.183012 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:52:39.183507 (+ 495us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:52:39.183511 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:52:39.183599 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:52:39.184176 (+ 577us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 02:52:39.184189 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0811 02:52:39.186224 (+ 2035us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.12.138.254:34683: Server oryx/127.12.138.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":309,"thread_start_us":195,"threads_started":1}
W20260811 02:52:39.186462 23647 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 02:52:39.180571 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.138.195:41399 (local address 127.12.138.254:34683)
0811 02:52:39.180735 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0811 02:52:39.180742 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 02:52:39.180962 (+ 220us) server_negotiation.cc:407] Connection header received
0811 02:52:39.181128 (+ 166us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:52:39.181132 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:52:39.181207 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:52:39.181299 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:52:39.181781 (+ 482us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:52:39.182327 (+ 546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:52:39.183127 (+ 800us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:52:39.183378 (+ 251us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:52:39.183492 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:52:39.186355 (+ 2863us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.12.138.195:41399: BlockingRecv error: recv got EOF from 127.12.138.195:41399 (error 108)
Metrics: {"server-negotiator.queue_time_us":60}
Aug 11 02:52:39 dist-test-slave-9gcw krb5kdc[22473](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.138.193@KRBTEST.COM for oryx/127.12.138.254@KRBTEST.COM, Server not found in Kerberos database
Aug 11 02:52:39 dist-test-slave-9gcw krb5kdc[22473](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.138.193@KRBTEST.COM for oryx/127.12.138.254@KRBTEST.COM, Server not found in Kerberos database
I20260811 02:52:39.776961 23650 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:52:39.770121 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.138.254:34683 (local address 127.12.138.193:39879)
0811 02:52:39.770525 (+ 404us) negotiation.cc:107] Waiting for socket to connect
0811 02:52:39.770540 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 02:52:39.770767 (+ 227us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:52:39.771086 (+ 319us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:52:39.771089 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:52:39.771311 (+ 222us) client_negotiation.cc:189] Negotiated authn=SASL
0811 02:52:39.771445 (+ 134us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:52:39.771455 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:52:39.772253 (+ 798us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:52:39.772256 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:52:39.773057 (+ 801us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:52:39.773071 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:52:39.773700 (+ 629us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:52:39.773704 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:52:39.773822 (+ 118us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:52:39.774511 (+ 689us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 02:52:39.774528 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 02:52:39.776838 (+ 2310us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.12.138.254:34683: Server oryx/127.12.138.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":313,"thread_start_us":93,"threads_started":1}
W20260811 02:52:39.777226 23651 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 02:52:39.770320 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.138.193:39879 (local address 127.12.138.254:34683)
0811 02:52:39.770581 (+ 261us) server_negotiation.cc:206] Beginning negotiation
0811 02:52:39.770584 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 02:52:39.770605 (+ 21us) server_negotiation.cc:407] Connection header received
0811 02:52:39.770891 (+ 286us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:52:39.770894 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:52:39.770944 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:52:39.771029 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:52:39.771573 (+ 544us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:52:39.772117 (+ 544us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:52:39.773314 (+ 1197us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:52:39.773569 (+ 255us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:52:39.773693 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:52:39.777046 (+ 3353us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.12.138.193:39879: BlockingRecv error: recv got EOF from 127.12.138.193:39879 (error 108)
Metrics: {"server-negotiator.queue_time_us":186,"thread_start_us":68,"threads_started":1}
Aug 11 02:52:39 dist-test-slave-9gcw krb5kdc[22473](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.138.194@KRBTEST.COM for oryx/127.12.138.254@KRBTEST.COM, Server not found in Kerberos database
Aug 11 02:52:39 dist-test-slave-9gcw krb5kdc[22473](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.138.194@KRBTEST.COM for oryx/127.12.138.254@KRBTEST.COM, Server not found in Kerberos database
I20260811 02:52:40.002265 23652 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:52:39.988592 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.138.254:34683 (local address 127.12.138.194:59119)
0811 02:52:39.988988 (+ 396us) negotiation.cc:107] Waiting for socket to connect
0811 02:52:39.989023 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0811 02:52:39.989206 (+ 183us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:52:39.989559 (+ 353us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:52:39.989562 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:52:39.989803 (+ 241us) client_negotiation.cc:189] Negotiated authn=SASL
0811 02:52:39.989905 (+ 102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:52:39.989911 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:52:39.990835 (+ 924us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:52:39.990838 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:52:39.991431 (+ 593us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:52:39.991440 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:52:39.991894 (+ 454us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:52:39.991898 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:52:39.991979 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:52:39.999475 (+ 7496us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 02:52:39.999490 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0811 02:52:40.002092 (+ 2602us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.12.138.254:34683: Server oryx/127.12.138.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":287,"thread_start_us":84,"threads_started":1}
W20260811 02:52:40.002494 23651 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 02:52:39.988689 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.138.194:59119 (local address 127.12.138.254:34683)
0811 02:52:39.988922 (+ 233us) server_negotiation.cc:206] Beginning negotiation
0811 02:52:39.988927 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 02:52:39.989125 (+ 198us) server_negotiation.cc:407] Connection header received
0811 02:52:39.989335 (+ 210us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:52:39.989338 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:52:39.989398 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:52:39.989493 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:52:39.990058 (+ 565us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:52:39.990718 (+ 660us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:52:39.991538 (+ 820us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:52:39.991753 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:52:39.991853 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:52:40.002244 (+ 10391us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.12.138.194:59119: BlockingRecv error: recv got EOF from 127.12.138.194:59119 (error 108)
Metrics: {"server-negotiator.queue_time_us":126}
Aug 11 02:52:40 dist-test-slave-9gcw krb5kdc[22473](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.138.195@KRBTEST.COM for oryx/127.12.138.254@KRBTEST.COM, Server not found in Kerberos database
Aug 11 02:52:40 dist-test-slave-9gcw krb5kdc[22473](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.138.195@KRBTEST.COM for oryx/127.12.138.254@KRBTEST.COM, Server not found in Kerberos database
I20260811 02:52:40.193287 23653 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:52:40.187294 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.138.254:34683 (local address 127.12.138.195:44167)
0811 02:52:40.187668 (+ 374us) negotiation.cc:107] Waiting for socket to connect
0811 02:52:40.187681 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 02:52:40.187820 (+ 139us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:52:40.188139 (+ 319us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:52:40.188142 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:52:40.188410 (+ 268us) client_negotiation.cc:189] Negotiated authn=SASL
0811 02:52:40.188509 (+ 99us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:52:40.188515 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:52:40.189354 (+ 839us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:52:40.189358 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:52:40.189918 (+ 560us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:52:40.189926 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:52:40.190491 (+ 565us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:52:40.190499 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:52:40.190607 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:52:40.191235 (+ 628us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 02:52:40.191248 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0811 02:52:40.193102 (+ 1854us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.12.138.254:34683: Server oryx/127.12.138.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":295,"thread_start_us":89,"threads_started":1}
W20260811 02:52:40.193445 23651 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 02:52:40.187430 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.138.195:44167 (local address 127.12.138.254:34683)
0811 02:52:40.187595 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0811 02:52:40.187599 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 02:52:40.187756 (+ 157us) server_negotiation.cc:407] Connection header received
0811 02:52:40.187938 (+ 182us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:52:40.187941 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:52:40.188000 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:52:40.188112 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:52:40.188630 (+ 518us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:52:40.189236 (+ 606us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:52:40.190039 (+ 803us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:52:40.190319 (+ 280us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:52:40.190427 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:52:40.193295 (+ 2868us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.12.138.195:44167: BlockingRecv error: recv got EOF from 127.12.138.195:44167 (error 108)
Metrics: {"server-negotiator.queue_time_us":66}
Aug 11 02:52:40 dist-test-slave-9gcw krb5kdc[22473](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.138.193@KRBTEST.COM for oryx/127.12.138.254@KRBTEST.COM, Server not found in Kerberos database
Aug 11 02:52:40 dist-test-slave-9gcw krb5kdc[22473](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.138.193@KRBTEST.COM for oryx/127.12.138.254@KRBTEST.COM, Server not found in Kerberos database
I20260811 02:52:40.784546 23655 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:52:40.778084 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.138.254:34683 (local address 127.12.138.193:39971)
0811 02:52:40.778427 (+ 343us) negotiation.cc:107] Waiting for socket to connect
0811 02:52:40.778443 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 02:52:40.778547 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:52:40.778920 (+ 373us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:52:40.778923 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:52:40.779132 (+ 209us) client_negotiation.cc:189] Negotiated authn=SASL
0811 02:52:40.779238 (+ 106us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:52:40.779243 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:52:40.780234 (+ 991us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:52:40.780237 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:52:40.780923 (+ 686us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:52:40.780933 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:52:40.781498 (+ 565us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:52:40.781501 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:52:40.781575 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:52:40.782118 (+ 543us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 02:52:40.782130 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0811 02:52:40.784423 (+ 2293us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.12.138.254:34683: Server oryx/127.12.138.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":239,"thread_start_us":133,"threads_started":1}
W20260811 02:52:40.784754 23657 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 02:52:40.778275 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.138.193:39971 (local address 127.12.138.254:34683)
0811 02:52:40.778541 (+ 266us) server_negotiation.cc:206] Beginning negotiation
0811 02:52:40.778546 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 02:52:40.778563 (+ 17us) server_negotiation.cc:407] Connection header received
0811 02:52:40.778629 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:52:40.778632 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:52:40.778737 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:52:40.778839 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:52:40.779397 (+ 558us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:52:40.780104 (+ 707us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:52:40.781060 (+ 956us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:52:40.781376 (+ 316us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:52:40.781473 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:52:40.784569 (+ 3096us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.12.138.193:39971: BlockingRecv error: recv got EOF from 127.12.138.193:39971 (error 108)
Metrics: {"server-negotiator.queue_time_us":196,"thread_start_us":105,"threads_started":1}
Aug 11 02:52:41 dist-test-slave-9gcw krb5kdc[22473](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.138.194@KRBTEST.COM for oryx/127.12.138.254@KRBTEST.COM, Server not found in Kerberos database
Aug 11 02:52:41 dist-test-slave-9gcw krb5kdc[22473](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.138.194@KRBTEST.COM for oryx/127.12.138.254@KRBTEST.COM, Server not found in Kerberos database
I20260811 02:52:41.009472 23658 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:52:41.003196 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.138.254:34683 (local address 127.12.138.194:60403)
0811 02:52:41.003480 (+ 284us) negotiation.cc:107] Waiting for socket to connect
0811 02:52:41.003494 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 02:52:41.003594 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:52:41.003845 (+ 251us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:52:41.003848 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:52:41.004053 (+ 205us) client_negotiation.cc:189] Negotiated authn=SASL
0811 02:52:41.004168 (+ 115us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:52:41.004175 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:52:41.005098 (+ 923us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:52:41.005102 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:52:41.005782 (+ 680us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:52:41.005791 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:52:41.006332 (+ 541us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:52:41.006335 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:52:41.006415 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:52:41.007024 (+ 609us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 02:52:41.007037 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0811 02:52:41.009282 (+ 2245us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.12.138.254:34683: Server oryx/127.12.138.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":193,"thread_start_us":81,"threads_started":1}
W20260811 02:52:41.009526 23657 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 02:52:41.003343 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.138.194:60403 (local address 127.12.138.254:34683)
0811 02:52:41.003517 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0811 02:52:41.003521 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 02:52:41.003540 (+ 19us) server_negotiation.cc:407] Connection header received
0811 02:52:41.003690 (+ 150us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:52:41.003692 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:52:41.003737 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:52:41.003816 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:52:41.004276 (+ 460us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:52:41.004963 (+ 687us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:52:41.005941 (+ 978us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:52:41.006186 (+ 245us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:52:41.006281 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:52:41.009417 (+ 3136us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.12.138.194:60403: BlockingRecv error: recv got EOF from 127.12.138.194:60403 (error 108)
Metrics: {"server-negotiator.queue_time_us":75}
Aug 11 02:52:41 dist-test-slave-9gcw krb5kdc[22473](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.138.195@KRBTEST.COM for oryx/127.12.138.254@KRBTEST.COM, Server not found in Kerberos database
Aug 11 02:52:41 dist-test-slave-9gcw krb5kdc[22473](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.138.195@KRBTEST.COM for oryx/127.12.138.254@KRBTEST.COM, Server not found in Kerberos database
I20260811 02:52:41.199237 23659 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:52:41.194009 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.138.254:34683 (local address 127.12.138.195:42737)
0811 02:52:41.194278 (+ 269us) negotiation.cc:107] Waiting for socket to connect
0811 02:52:41.194293 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 02:52:41.194396 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:52:41.194643 (+ 247us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:52:41.194645 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:52:41.194856 (+ 211us) client_negotiation.cc:189] Negotiated authn=SASL
0811 02:52:41.194954 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:52:41.194959 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:52:41.195719 (+ 760us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:52:41.195721 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:52:41.196284 (+ 563us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:52:41.196292 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:52:41.196725 (+ 433us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:52:41.196728 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:52:41.196788 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:52:41.197330 (+ 542us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 02:52:41.197342 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0811 02:52:41.199120 (+ 1778us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.12.138.254:34683: Server oryx/127.12.138.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":184,"thread_start_us":75,"threads_started":1}
W20260811 02:52:41.199433 23657 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 02:52:41.194137 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.138.195:42737 (local address 127.12.138.254:34683)
0811 02:52:41.194299 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0811 02:52:41.194303 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 02:52:41.194360 (+ 57us) server_negotiation.cc:407] Connection header received
0811 02:52:41.194488 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:52:41.194490 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:52:41.194538 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:52:41.194607 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:52:41.195049 (+ 442us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:52:41.195626 (+ 577us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:52:41.196392 (+ 766us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:52:41.196620 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:52:41.196723 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:52:41.199268 (+ 2545us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.12.138.195:42737: BlockingRecv error: recv got EOF from 127.12.138.195:42737 (error 108)
Metrics: {"server-negotiator.queue_time_us":67}
Aug 11 02:52:41 dist-test-slave-9gcw krb5kdc[22473](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.138.193@KRBTEST.COM for oryx/127.12.138.254@KRBTEST.COM, Server not found in Kerberos database
Aug 11 02:52:41 dist-test-slave-9gcw krb5kdc[22473](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.138.193@KRBTEST.COM for oryx/127.12.138.254@KRBTEST.COM, Server not found in Kerberos database
I20260811 02:52:41.791919 23660 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:52:41.785424 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.138.254:34683 (local address 127.12.138.193:36353)
0811 02:52:41.785780 (+ 356us) negotiation.cc:107] Waiting for socket to connect
0811 02:52:41.785803 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0811 02:52:41.785962 (+ 159us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:52:41.786188 (+ 226us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:52:41.786190 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:52:41.786392 (+ 202us) client_negotiation.cc:189] Negotiated authn=SASL
0811 02:52:41.786500 (+ 108us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:52:41.786506 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:52:41.787361 (+ 855us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:52:41.787364 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:52:41.788132 (+ 768us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:52:41.788145 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:52:41.788647 (+ 502us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:52:41.788651 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:52:41.788837 (+ 186us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:52:41.789560 (+ 723us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 02:52:41.789573 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0811 02:52:41.791800 (+ 2227us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.12.138.254:34683: Server oryx/127.12.138.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":243,"thread_start_us":101,"threads_started":1}
W20260811 02:52:41.792150 23661 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 02:52:41.785544 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.138.193:36353 (local address 127.12.138.254:34683)
0811 02:52:41.785967 (+ 423us) server_negotiation.cc:206] Beginning negotiation
0811 02:52:41.785971 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 02:52:41.785993 (+ 22us) server_negotiation.cc:407] Connection header received
0811 02:52:41.786035 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:52:41.786037 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:52:41.786078 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:52:41.786163 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:52:41.786628 (+ 465us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:52:41.787250 (+ 622us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:52:41.788269 (+ 1019us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:52:41.788499 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:52:41.788581 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:52:41.791971 (+ 3390us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.12.138.193:36353: BlockingRecv error: recv got EOF from 127.12.138.193:36353 (error 108)
Metrics: {"server-negotiator.queue_time_us":340,"thread_start_us":98,"threads_started":1}
Aug 11 02:52:42 dist-test-slave-9gcw krb5kdc[22473](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.138.194@KRBTEST.COM for oryx/127.12.138.254@KRBTEST.COM, Server not found in Kerberos database
Aug 11 02:52:42 dist-test-slave-9gcw krb5kdc[22473](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.138.194@KRBTEST.COM for oryx/127.12.138.254@KRBTEST.COM, Server not found in Kerberos database
I20260811 02:52:42.015988 23662 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:52:42.010373 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.138.254:34683 (local address 127.12.138.194:49705)
0811 02:52:42.010659 (+ 286us) negotiation.cc:107] Waiting for socket to connect
0811 02:52:42.010674 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 02:52:42.010787 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:52:42.011050 (+ 263us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:52:42.011052 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:52:42.011246 (+ 194us) client_negotiation.cc:189] Negotiated authn=SASL
0811 02:52:42.011344 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:52:42.011349 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:52:42.012216 (+ 867us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:52:42.012218 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:52:42.012758 (+ 540us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:52:42.012764 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:52:42.013305 (+ 541us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:52:42.013308 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:52:42.013376 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:52:42.013892 (+ 516us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 02:52:42.013903 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0811 02:52:42.015844 (+ 1941us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.12.138.254:34683: Server oryx/127.12.138.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":199,"thread_start_us":113,"threads_started":1}
W20260811 02:52:42.016235 23661 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 02:52:42.010545 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.138.194:49705 (local address 127.12.138.254:34683)
0811 02:52:42.010697 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0811 02:52:42.010704 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 02:52:42.010727 (+ 23us) server_negotiation.cc:407] Connection header received
0811 02:52:42.010907 (+ 180us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:52:42.010910 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:52:42.010960 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:52:42.011039 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:52:42.011457 (+ 418us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:52:42.012089 (+ 632us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:52:42.012885 (+ 796us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:52:42.013189 (+ 304us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:52:42.013280 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:52:42.016078 (+ 2798us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.12.138.194:49705: BlockingRecv error: recv got EOF from 127.12.138.194:49705 (error 108)
Metrics: {"server-negotiator.queue_time_us":55}
Aug 11 02:52:42 dist-test-slave-9gcw krb5kdc[22473](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.138.195@KRBTEST.COM for oryx/127.12.138.254@KRBTEST.COM, Server not found in Kerberos database
Aug 11 02:52:42 dist-test-slave-9gcw krb5kdc[22473](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.138.195@KRBTEST.COM for oryx/127.12.138.254@KRBTEST.COM, Server not found in Kerberos database
I20260811 02:52:42.206878 23664 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:52:42.200033 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.138.254:34683 (local address 127.12.138.195:36657)
0811 02:52:42.200443 (+ 410us) negotiation.cc:107] Waiting for socket to connect
0811 02:52:42.200462 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0811 02:52:42.200732 (+ 270us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:52:42.201111 (+ 379us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:52:42.201115 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:52:42.201451 (+ 336us) client_negotiation.cc:189] Negotiated authn=SASL
0811 02:52:42.201620 (+ 169us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:52:42.201650 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:52:42.202423 (+ 773us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:52:42.202426 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:52:42.203372 (+ 946us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:52:42.203384 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:52:42.203831 (+ 447us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:52:42.203835 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:52:42.203907 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:52:42.204440 (+ 533us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 02:52:42.204452 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0811 02:52:42.206710 (+ 2258us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.12.138.254:34683: Server oryx/127.12.138.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":318,"thread_start_us":121,"threads_started":1}
W20260811 02:52:42.206934 23661 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 02:52:42.200186 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.138.195:36657 (local address 127.12.138.254:34683)
0811 02:52:42.200395 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0811 02:52:42.200401 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 02:52:42.200566 (+ 165us) server_negotiation.cc:407] Connection header received
0811 02:52:42.200886 (+ 320us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:52:42.200890 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:52:42.200954 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:52:42.201095 (+ 141us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:52:42.201773 (+ 678us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:52:42.202306 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:52:42.203474 (+ 1168us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:52:42.203696 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:52:42.203778 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:52:42.206839 (+ 3061us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.12.138.195:36657: BlockingRecv error: recv got EOF from 127.12.138.195:36657 (error 108)
Metrics: {"server-negotiator.queue_time_us":89}
Aug 11 02:52:42 dist-test-slave-9gcw krb5kdc[22473](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.138.193@KRBTEST.COM for oryx/127.12.138.254@KRBTEST.COM, Server not found in Kerberos database
Aug 11 02:52:42 dist-test-slave-9gcw krb5kdc[22473](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.138.193@KRBTEST.COM for oryx/127.12.138.254@KRBTEST.COM, Server not found in Kerberos database
I20260811 02:52:42.798810 23665 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:52:42.792820 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.138.254:34683 (local address 127.12.138.193:49407)
0811 02:52:42.793216 (+ 396us) negotiation.cc:107] Waiting for socket to connect
0811 02:52:42.793238 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0811 02:52:42.793400 (+ 162us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:52:42.793774 (+ 374us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:52:42.793776 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:52:42.793986 (+ 210us) client_negotiation.cc:189] Negotiated authn=SASL
0811 02:52:42.794095 (+ 109us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:52:42.794100 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:52:42.794888 (+ 788us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:52:42.794891 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:52:42.795518 (+ 627us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:52:42.795528 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:52:42.795992 (+ 464us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:52:42.795995 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:52:42.796058 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:52:42.796572 (+ 514us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 02:52:42.796583 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0811 02:52:42.798608 (+ 2025us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.12.138.254:34683: Server oryx/127.12.138.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":283,"thread_start_us":118,"threads_started":1}
W20260811 02:52:42.798872 23666 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 02:52:42.792954 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.138.193:49407 (local address 127.12.138.254:34683)
0811 02:52:42.793287 (+ 333us) server_negotiation.cc:206] Beginning negotiation
0811 02:52:42.793319 (+ 32us) server_negotiation.cc:399] Waiting for connection header
0811 02:52:42.793333 (+ 14us) server_negotiation.cc:407] Connection header received
0811 02:52:42.793528 (+ 195us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:52:42.793535 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:52:42.793582 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:52:42.793661 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:52:42.794207 (+ 546us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:52:42.794758 (+ 551us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:52:42.795639 (+ 881us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:52:42.795885 (+ 246us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:52:42.795968 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:52:42.798750 (+ 2782us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.12.138.193:49407: BlockingRecv error: recv got EOF from 127.12.138.193:49407 (error 108)
Metrics: {"server-negotiator.queue_time_us":235,"thread_start_us":64,"threads_started":1}
Aug 11 02:52:43 dist-test-slave-9gcw krb5kdc[22473](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.138.194@KRBTEST.COM for oryx/127.12.138.254@KRBTEST.COM, Server not found in Kerberos database
Aug 11 02:52:43 dist-test-slave-9gcw krb5kdc[22473](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.138.194@KRBTEST.COM for oryx/127.12.138.254@KRBTEST.COM, Server not found in Kerberos database
I20260811 02:52:43.022768 23667 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:52:43.017019 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.138.254:34683 (local address 127.12.138.194:33489)
0811 02:52:43.017330 (+ 311us) negotiation.cc:107] Waiting for socket to connect
0811 02:52:43.017343 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 02:52:43.017477 (+ 134us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:52:43.017756 (+ 279us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:52:43.017759 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:52:43.017998 (+ 239us) client_negotiation.cc:189] Negotiated authn=SASL
0811 02:52:43.018095 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:52:43.018101 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:52:43.018878 (+ 777us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:52:43.018883 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:52:43.019447 (+ 564us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:52:43.019454 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:52:43.019940 (+ 486us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:52:43.019944 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:52:43.020023 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:52:43.020614 (+ 591us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 02:52:43.020625 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0811 02:52:43.022594 (+ 1969us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.12.138.254:34683: Server oryx/127.12.138.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":229,"thread_start_us":112,"threads_started":1}
W20260811 02:52:43.022837 23666 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 02:52:43.017061 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.138.194:33489 (local address 127.12.138.254:34683)
0811 02:52:43.017249 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0811 02:52:43.017255 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 02:52:43.017422 (+ 167us) server_negotiation.cc:407] Connection header received
0811 02:52:43.017581 (+ 159us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:52:43.017585 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:52:43.017634 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:52:43.017737 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:52:43.018220 (+ 483us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:52:43.018763 (+ 543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:52:43.019579 (+ 816us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:52:43.019827 (+ 248us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:52:43.019940 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:52:43.022726 (+ 2786us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.12.138.194:33489: BlockingRecv error: recv got EOF from 127.12.138.194:33489 (error 108)
Metrics: {"server-negotiator.queue_time_us":57}
Aug 11 02:52:43 dist-test-slave-9gcw krb5kdc[22473](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.138.195@KRBTEST.COM for oryx/127.12.138.254@KRBTEST.COM, Server not found in Kerberos database
Aug 11 02:52:43 dist-test-slave-9gcw krb5kdc[22473](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.138.195@KRBTEST.COM for oryx/127.12.138.254@KRBTEST.COM, Server not found in Kerberos database
I20260811 02:52:43.214331 23668 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:52:43.207883 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.138.254:34683 (local address 127.12.138.195:48779)
0811 02:52:43.208273 (+ 390us) negotiation.cc:107] Waiting for socket to connect
0811 02:52:43.208294 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0811 02:52:43.208424 (+ 130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:52:43.208824 (+ 400us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:52:43.208827 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:52:43.209090 (+ 263us) client_negotiation.cc:189] Negotiated authn=SASL
0811 02:52:43.209236 (+ 146us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:52:43.209243 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:52:43.210133 (+ 890us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:52:43.210148 (+ 15us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:52:43.210777 (+ 629us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:52:43.210786 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:52:43.211315 (+ 529us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:52:43.211319 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:52:43.211415 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:52:43.211971 (+ 556us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 02:52:43.211984 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0811 02:52:43.214097 (+ 2113us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.12.138.254:34683: Server oryx/127.12.138.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":245,"thread_start_us":134,"threads_started":1}
W20260811 02:52:43.214375 23666 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 02:52:43.208106 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.138.195:48779 (local address 127.12.138.254:34683)
0811 02:52:43.208279 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0811 02:52:43.208287 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 02:52:43.208385 (+ 98us) server_negotiation.cc:407] Connection header received
0811 02:52:43.208555 (+ 170us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:52:43.208558 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:52:43.208622 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:52:43.208803 (+ 181us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:52:43.209378 (+ 575us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:52:43.209997 (+ 619us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:52:43.210917 (+ 920us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:52:43.211185 (+ 268us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:52:43.211309 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:52:43.214262 (+ 2953us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.12.138.195:48779: BlockingRecv error: recv got EOF from 127.12.138.195:48779 (error 108)
Metrics: {"server-negotiator.queue_time_us":91}
Aug 11 02:52:43 dist-test-slave-9gcw krb5kdc[22473](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.138.193@KRBTEST.COM for oryx/127.12.138.254@KRBTEST.COM, Server not found in Kerberos database
Aug 11 02:52:43 dist-test-slave-9gcw krb5kdc[22473](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.138.193@KRBTEST.COM for oryx/127.12.138.254@KRBTEST.COM, Server not found in Kerberos database
I20260811 02:52:43.806358 23669 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:52:43.799940 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.138.254:34683 (local address 127.12.138.193:35861)
0811 02:52:43.800284 (+ 344us) negotiation.cc:107] Waiting for socket to connect
0811 02:52:43.800302 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 02:52:43.800435 (+ 133us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:52:43.800747 (+ 312us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:52:43.800752 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:52:43.801058 (+ 306us) client_negotiation.cc:189] Negotiated authn=SASL
0811 02:52:43.801248 (+ 190us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:52:43.801260 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:52:43.802117 (+ 857us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:52:43.802120 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:52:43.802788 (+ 668us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:52:43.802797 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:52:43.803258 (+ 461us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:52:43.803261 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:52:43.803331 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:52:43.803899 (+ 568us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 02:52:43.803912 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0811 02:52:43.806181 (+ 2269us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.12.138.254:34683: Server oryx/127.12.138.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":214,"thread_start_us":115,"threads_started":1}
W20260811 02:52:43.806471 23670 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 02:52:43.800033 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.138.193:35861 (local address 127.12.138.254:34683)
0811 02:52:43.800286 (+ 253us) server_negotiation.cc:206] Beginning negotiation
0811 02:52:43.800302 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0811 02:52:43.800388 (+ 86us) server_negotiation.cc:407] Connection header received
0811 02:52:43.800549 (+ 161us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:52:43.800551 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:52:43.800598 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:52:43.800678 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:52:43.801416 (+ 738us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:52:43.802005 (+ 589us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:52:43.802906 (+ 901us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:52:43.803137 (+ 231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:52:43.803215 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:52:43.806329 (+ 3114us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.12.138.193:35861: BlockingRecv error: recv got EOF from 127.12.138.193:35861 (error 108)
Metrics: {"server-negotiator.queue_time_us":173,"thread_start_us":89,"threads_started":1}
Aug 11 02:52:44 dist-test-slave-9gcw krb5kdc[22473](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.138.194@KRBTEST.COM for oryx/127.12.138.254@KRBTEST.COM, Server not found in Kerberos database
Aug 11 02:52:44 dist-test-slave-9gcw krb5kdc[22473](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.138.194@KRBTEST.COM for oryx/127.12.138.254@KRBTEST.COM, Server not found in Kerberos database
I20260811 02:52:44.029966 23671 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:52:44.023632 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.138.254:34683 (local address 127.12.138.194:36965)
0811 02:52:44.023956 (+ 324us) negotiation.cc:107] Waiting for socket to connect
0811 02:52:44.023969 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 02:52:44.024122 (+ 153us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:52:44.024414 (+ 292us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:52:44.024418 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:52:44.024653 (+ 235us) client_negotiation.cc:189] Negotiated authn=SASL
0811 02:52:44.024754 (+ 101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:52:44.024760 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:52:44.025607 (+ 847us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:52:44.025612 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:52:44.026178 (+ 566us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:52:44.026186 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:52:44.026658 (+ 472us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:52:44.026662 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:52:44.026751 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:52:44.027508 (+ 757us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 02:52:44.027527 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0811 02:52:44.029846 (+ 2319us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.12.138.254:34683: Server oryx/127.12.138.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":238,"thread_start_us":81,"threads_started":1}
W20260811 02:52:44.030215 23670 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 02:52:44.023717 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.138.194:36965 (local address 127.12.138.254:34683)
0811 02:52:44.023892 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0811 02:52:44.023898 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 02:52:44.024050 (+ 152us) server_negotiation.cc:407] Connection header received
0811 02:52:44.024231 (+ 181us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:52:44.024235 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:52:44.024289 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:52:44.024388 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:52:44.024883 (+ 495us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:52:44.025490 (+ 607us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:52:44.026311 (+ 821us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:52:44.026541 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:52:44.026653 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:52:44.030070 (+ 3417us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.12.138.194:36965: BlockingRecv error: recv got EOF from 127.12.138.194:36965 (error 108)
Metrics: {"server-negotiator.queue_time_us":50}
Aug 11 02:52:44 dist-test-slave-9gcw krb5kdc[22473](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.138.195@KRBTEST.COM for oryx/127.12.138.254@KRBTEST.COM, Server not found in Kerberos database
Aug 11 02:52:44 dist-test-slave-9gcw krb5kdc[22473](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.138.195@KRBTEST.COM for oryx/127.12.138.254@KRBTEST.COM, Server not found in Kerberos database
I20260811 02:52:44.221560 23672 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:52:44.215278 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.138.254:34683 (local address 127.12.138.195:44851)
0811 02:52:44.215606 (+ 328us) negotiation.cc:107] Waiting for socket to connect
0811 02:52:44.215619 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 02:52:44.215744 (+ 125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:52:44.216013 (+ 269us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:52:44.216015 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:52:44.216230 (+ 215us) client_negotiation.cc:189] Negotiated authn=SASL
0811 02:52:44.216327 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:52:44.216332 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:52:44.217302 (+ 970us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:52:44.217305 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:52:44.217887 (+ 582us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:52:44.217897 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:52:44.218415 (+ 518us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:52:44.218419 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:52:44.218508 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:52:44.219152 (+ 644us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 02:52:44.219169 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 02:52:44.221443 (+ 2274us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.12.138.254:34683: Server oryx/127.12.138.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":235,"thread_start_us":149,"threads_started":1}
W20260811 02:52:44.221817 23670 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 02:52:44.215442 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.138.195:44851 (local address 127.12.138.254:34683)
0811 02:52:44.215595 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0811 02:52:44.215599 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 02:52:44.215731 (+ 132us) server_negotiation.cc:407] Connection header received
0811 02:52:44.215839 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:52:44.215841 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:52:44.215891 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:52:44.215957 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:52:44.216474 (+ 517us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:52:44.217105 (+ 631us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:52:44.218023 (+ 918us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:52:44.218264 (+ 241us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:52:44.218357 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:52:44.221593 (+ 3236us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.12.138.195:44851: BlockingRecv error: recv got EOF from 127.12.138.195:44851 (error 108)
Metrics: {"server-negotiator.queue_time_us":57}
Aug 11 02:52:44 dist-test-slave-9gcw krb5kdc[22473](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.138.193@KRBTEST.COM for oryx/127.12.138.254@KRBTEST.COM, Server not found in Kerberos database
Aug 11 02:52:44 dist-test-slave-9gcw krb5kdc[22473](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.138.193@KRBTEST.COM for oryx/127.12.138.254@KRBTEST.COM, Server not found in Kerberos database
I20260811 02:52:44.813108 23673 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:52:44.807241 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.138.254:34683 (local address 127.12.138.193:56857)
0811 02:52:44.807569 (+ 328us) negotiation.cc:107] Waiting for socket to connect
0811 02:52:44.807621 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0811 02:52:44.807810 (+ 189us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:52:44.808063 (+ 253us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:52:44.808065 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:52:44.808261 (+ 196us) client_negotiation.cc:189] Negotiated authn=SASL
0811 02:52:44.808362 (+ 101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:52:44.808366 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:52:44.809133 (+ 767us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:52:44.809136 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:52:44.809780 (+ 644us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:52:44.809789 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:52:44.810236 (+ 447us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:52:44.810239 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:52:44.810303 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:52:44.811038 (+ 735us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 02:52:44.811050 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0811 02:52:44.812982 (+ 1932us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.12.138.254:34683: Server oryx/127.12.138.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":230,"thread_start_us":113,"threads_started":1}
W20260811 02:52:44.813360 23674 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 02:52:44.807413 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.138.193:56857 (local address 127.12.138.254:34683)
0811 02:52:44.807681 (+ 268us) server_negotiation.cc:206] Beginning negotiation
0811 02:52:44.807685 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 02:52:44.807768 (+ 83us) server_negotiation.cc:407] Connection header received
0811 02:52:44.807896 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:52:44.807898 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:52:44.807945 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:52:44.808024 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:52:44.808468 (+ 444us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:52:44.809026 (+ 558us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:52:44.809890 (+ 864us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:52:44.810113 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:52:44.810192 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:52:44.813131 (+ 2939us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.12.138.193:56857: BlockingRecv error: recv got EOF from 127.12.138.193:56857 (error 108)
Metrics: {"server-negotiator.queue_time_us":192,"thread_start_us":85,"threads_started":1}
Aug 11 02:52:45 dist-test-slave-9gcw krb5kdc[22473](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.138.194@KRBTEST.COM for oryx/127.12.138.254@KRBTEST.COM, Server not found in Kerberos database
Aug 11 02:52:45 dist-test-slave-9gcw krb5kdc[22473](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.138.194@KRBTEST.COM for oryx/127.12.138.254@KRBTEST.COM, Server not found in Kerberos database
I20260811 02:52:45.042582 23675 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:52:45.030894 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.138.254:34683 (local address 127.12.138.194:60345)
0811 02:52:45.031174 (+ 280us) negotiation.cc:107] Waiting for socket to connect
0811 02:52:45.031192 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 02:52:45.031312 (+ 120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:52:45.031597 (+ 285us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:52:45.031599 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:52:45.031807 (+ 208us) client_negotiation.cc:189] Negotiated authn=SASL
0811 02:52:45.031907 (+ 100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:52:45.031912 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:52:45.032867 (+ 955us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:52:45.032870 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:52:45.033537 (+ 667us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:52:45.033547 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:52:45.034100 (+ 553us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:52:45.034102 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:52:45.034166 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:52:45.040003 (+ 5837us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 02:52:45.040019 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0811 02:52:45.042394 (+ 2375us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.12.138.254:34683: Server oryx/127.12.138.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":194,"thread_start_us":102,"threads_started":1}
W20260811 02:52:45.042742 23674 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 02:52:45.031191 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.138.194:60345 (local address 127.12.138.254:34683)
0811 02:52:45.031337 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0811 02:52:45.031343 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 02:52:45.031361 (+ 18us) server_negotiation.cc:407] Connection header received
0811 02:52:45.031406 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:52:45.031409 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:52:45.031460 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:52:45.031538 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:52:45.032062 (+ 524us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:52:45.032737 (+ 675us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:52:45.033662 (+ 925us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:52:45.033983 (+ 321us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:52:45.034081 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:52:45.042604 (+ 8523us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.12.138.194:60345: BlockingRecv error: recv got EOF from 127.12.138.194:60345 (error 108)
Metrics: {"server-negotiator.queue_time_us":37}
Aug 11 02:52:45 dist-test-slave-9gcw krb5kdc[22473](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.138.195@KRBTEST.COM for oryx/127.12.138.254@KRBTEST.COM, Server not found in Kerberos database
Aug 11 02:52:45 dist-test-slave-9gcw krb5kdc[22473](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.138.195@KRBTEST.COM for oryx/127.12.138.254@KRBTEST.COM, Server not found in Kerberos database
I20260811 02:52:45.228243 23676 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:52:45.222350 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.138.254:34683 (local address 127.12.138.195:33325)
0811 02:52:45.222721 (+ 371us) negotiation.cc:107] Waiting for socket to connect
0811 02:52:45.222734 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 02:52:45.222877 (+ 143us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:52:45.223178 (+ 301us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:52:45.223180 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:52:45.223395 (+ 215us) client_negotiation.cc:189] Negotiated authn=SASL
0811 02:52:45.223496 (+ 101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:52:45.223501 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:52:45.224345 (+ 844us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:52:45.224347 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:52:45.224893 (+ 546us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:52:45.224900 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:52:45.225395 (+ 495us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:52:45.225398 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:52:45.225462 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:52:45.226047 (+ 585us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 02:52:45.226061 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0811 02:52:45.228120 (+ 2059us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.12.138.254:34683: Server oryx/127.12.138.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":294,"thread_start_us":109,"threads_started":1}
W20260811 02:52:45.228423 23674 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 02:52:45.222507 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.138.195:33325 (local address 127.12.138.254:34683)
0811 02:52:45.222757 (+ 250us) server_negotiation.cc:206] Beginning negotiation
0811 02:52:45.222764 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 02:52:45.222814 (+ 50us) server_negotiation.cc:407] Connection header received
0811 02:52:45.222998 (+ 184us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:52:45.223001 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:52:45.223055 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:52:45.223141 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:52:45.223678 (+ 537us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:52:45.224235 (+ 557us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:52:45.225005 (+ 770us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:52:45.225290 (+ 285us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:52:45.225386 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:52:45.228256 (+ 2870us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.12.138.195:33325: BlockingRecv error: recv got EOF from 127.12.138.195:33325 (error 108)
Metrics: {"server-negotiator.queue_time_us":68}
Aug 11 02:52:45 dist-test-slave-9gcw krb5kdc[22473](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.138.193@KRBTEST.COM for oryx/127.12.138.254@KRBTEST.COM, Server not found in Kerberos database
Aug 11 02:52:45 dist-test-slave-9gcw krb5kdc[22473](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.138.193@KRBTEST.COM for oryx/127.12.138.254@KRBTEST.COM, Server not found in Kerberos database
I20260811 02:52:45.820619 23678 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:52:45.814074 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.138.254:34683 (local address 127.12.138.193:49127)
0811 02:52:45.814390 (+ 316us) negotiation.cc:107] Waiting for socket to connect
0811 02:52:45.814406 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 02:52:45.814505 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:52:45.814852 (+ 347us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:52:45.814855 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:52:45.815068 (+ 213us) client_negotiation.cc:189] Negotiated authn=SASL
0811 02:52:45.815225 (+ 157us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:52:45.815231 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:52:45.816229 (+ 998us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:52:45.816232 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:52:45.816896 (+ 664us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:52:45.816906 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:52:45.817574 (+ 668us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:52:45.817577 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:52:45.817648 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:52:45.818285 (+ 637us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 02:52:45.818298 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0811 02:52:45.820479 (+ 2181us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.12.138.254:34683: Server oryx/127.12.138.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":221,"thread_start_us":126,"threads_started":1}
W20260811 02:52:45.821108 23679 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 02:52:45.814226 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.138.193:49127 (local address 127.12.138.254:34683)
0811 02:52:45.814488 (+ 262us) server_negotiation.cc:206] Beginning negotiation
0811 02:52:45.814492 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 02:52:45.814508 (+ 16us) server_negotiation.cc:407] Connection header received
0811 02:52:45.814598 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:52:45.814601 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:52:45.814708 (+ 107us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:52:45.814795 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:52:45.815391 (+ 596us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:52:45.816101 (+ 710us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:52:45.817075 (+ 974us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:52:45.817442 (+ 367us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:52:45.817540 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:52:45.820939 (+ 3399us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.12.138.193:49127: BlockingRecv error: recv got EOF from 127.12.138.193:49127 (error 108)
Metrics: {"server-negotiator.queue_time_us":179,"thread_start_us":95,"threads_started":1}
Aug 11 02:52:46 dist-test-slave-9gcw krb5kdc[22473](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.138.194@KRBTEST.COM for oryx/127.12.138.254@KRBTEST.COM, Server not found in Kerberos database
Aug 11 02:52:46 dist-test-slave-9gcw krb5kdc[22473](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.138.194@KRBTEST.COM for oryx/127.12.138.254@KRBTEST.COM, Server not found in Kerberos database
I20260811 02:52:46.049587 23680 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:52:46.043590 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.138.254:34683 (local address 127.12.138.194:39645)
0811 02:52:46.043915 (+ 325us) negotiation.cc:107] Waiting for socket to connect
0811 02:52:46.043929 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 02:52:46.044033 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:52:46.044451 (+ 418us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:52:46.044453 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:52:46.044647 (+ 194us) client_negotiation.cc:189] Negotiated authn=SASL
0811 02:52:46.044762 (+ 115us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:52:46.044768 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:52:46.045716 (+ 948us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:52:46.045722 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:52:46.046305 (+ 583us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:52:46.046313 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:52:46.046791 (+ 478us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:52:46.046795 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:52:46.046883 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:52:46.047469 (+ 586us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 02:52:46.047481 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0811 02:52:46.049413 (+ 1932us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.12.138.254:34683: Server oryx/127.12.138.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":236,"thread_start_us":136,"threads_started":1}
W20260811 02:52:46.049649 23679 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 02:52:46.043750 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.138.194:39645 (local address 127.12.138.254:34683)
0811 02:52:46.043902 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0811 02:52:46.043906 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 02:52:46.043999 (+ 93us) server_negotiation.cc:407] Connection header received
0811 02:52:46.044194 (+ 195us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:52:46.044197 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:52:46.044313 (+ 116us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:52:46.044404 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:52:46.044919 (+ 515us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:52:46.045596 (+ 677us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:52:46.046442 (+ 846us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:52:46.046673 (+ 231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:52:46.046786 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:52:46.049540 (+ 2754us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.12.138.194:39645: BlockingRecv error: recv got EOF from 127.12.138.194:39645 (error 108)
Metrics: {"server-negotiator.queue_time_us":51}
Aug 11 02:52:46 dist-test-slave-9gcw krb5kdc[22473](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.138.195@KRBTEST.COM for oryx/127.12.138.254@KRBTEST.COM, Server not found in Kerberos database
Aug 11 02:52:46 dist-test-slave-9gcw krb5kdc[22473](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.138.195@KRBTEST.COM for oryx/127.12.138.254@KRBTEST.COM, Server not found in Kerberos database
I20260811 02:52:46.234746 23681 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:52:46.229082 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.138.254:34683 (local address 127.12.138.195:36771)
0811 02:52:46.229497 (+ 415us) negotiation.cc:107] Waiting for socket to connect
0811 02:52:46.229511 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 02:52:46.229615 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:52:46.229877 (+ 262us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:52:46.229880 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:52:46.230069 (+ 189us) client_negotiation.cc:189] Negotiated authn=SASL
0811 02:52:46.230164 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:52:46.230169 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:52:46.230895 (+ 726us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:52:46.230897 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:52:46.231436 (+ 539us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:52:46.231444 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:52:46.231871 (+ 427us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:52:46.231873 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:52:46.231935 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:52:46.232458 (+ 523us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 02:52:46.232469 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0811 02:52:46.234605 (+ 2136us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.12.138.254:34683: Server oryx/127.12.138.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":337,"thread_start_us":212,"threads_started":1}
W20260811 02:52:46.235147 23679 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 02:52:46.229191 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.138.195:36771 (local address 127.12.138.254:34683)
0811 02:52:46.229376 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0811 02:52:46.229381 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 02:52:46.229586 (+ 205us) server_negotiation.cc:407] Connection header received
0811 02:52:46.229685 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:52:46.229687 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:52:46.229738 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:52:46.229840 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:52:46.230264 (+ 424us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:52:46.230802 (+ 538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:52:46.231536 (+ 734us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:52:46.231750 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:52:46.231848 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:52:46.234990 (+ 3142us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.12.138.195:36771: BlockingRecv error: recv got EOF from 127.12.138.195:36771 (error 108)
Metrics: {"server-negotiator.queue_time_us":78}
Aug 11 02:52:46 dist-test-slave-9gcw krb5kdc[22473](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.138.193@KRBTEST.COM for oryx/127.12.138.254@KRBTEST.COM, Server not found in Kerberos database
Aug 11 02:52:46 dist-test-slave-9gcw krb5kdc[22473](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.138.193@KRBTEST.COM for oryx/127.12.138.254@KRBTEST.COM, Server not found in Kerberos database
I20260811 02:52:46.827615 23682 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:52:46.821525 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.138.254:34683 (local address 127.12.138.193:56371)
0811 02:52:46.821867 (+ 342us) negotiation.cc:107] Waiting for socket to connect
0811 02:52:46.821881 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 02:52:46.821976 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:52:46.822319 (+ 343us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:52:46.822321 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:52:46.822508 (+ 187us) client_negotiation.cc:189] Negotiated authn=SASL
0811 02:52:46.822695 (+ 187us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:52:46.822703 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:52:46.823574 (+ 871us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:52:46.823576 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:52:46.824196 (+ 620us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:52:46.824205 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:52:46.824705 (+ 500us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:52:46.824708 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:52:46.824776 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:52:46.825474 (+ 698us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 02:52:46.825486 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0811 02:52:46.827495 (+ 2009us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.12.138.254:34683: Server oryx/127.12.138.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":260,"thread_start_us":155,"threads_started":1}
W20260811 02:52:46.827898 23683 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 02:52:46.821726 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.138.193:56371 (local address 127.12.138.254:34683)
0811 02:52:46.822075 (+ 349us) server_negotiation.cc:206] Beginning negotiation
0811 02:52:46.822079 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 02:52:46.822109 (+ 30us) server_negotiation.cc:407] Connection header received
0811 02:52:46.822150 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:52:46.822152 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:52:46.822197 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:52:46.822284 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:52:46.822892 (+ 608us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:52:46.823467 (+ 575us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:52:46.824330 (+ 863us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:52:46.824571 (+ 241us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:52:46.824680 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:52:46.827736 (+ 3056us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.12.138.193:56371: BlockingRecv error: recv got EOF from 127.12.138.193:56371 (error 108)
Metrics: {"server-negotiator.queue_time_us":277,"thread_start_us":98,"threads_started":1}
Aug 11 02:52:47 dist-test-slave-9gcw krb5kdc[22473](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.138.194@KRBTEST.COM for oryx/127.12.138.254@KRBTEST.COM, Server not found in Kerberos database
Aug 11 02:52:47 dist-test-slave-9gcw krb5kdc[22473](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.138.194@KRBTEST.COM for oryx/127.12.138.254@KRBTEST.COM, Server not found in Kerberos database
I20260811 02:52:47.056763 23684 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:52:47.050461 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.138.254:34683 (local address 127.12.138.194:56465)
0811 02:52:47.050785 (+ 324us) negotiation.cc:107] Waiting for socket to connect
0811 02:52:47.050801 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 02:52:47.050938 (+ 137us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:52:47.051187 (+ 249us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:52:47.051189 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:52:47.051402 (+ 213us) client_negotiation.cc:189] Negotiated authn=SASL
0811 02:52:47.051503 (+ 101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:52:47.051508 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:52:47.052445 (+ 937us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:52:47.052448 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:52:47.053051 (+ 603us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:52:47.053059 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:52:47.053687 (+ 628us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:52:47.053690 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:52:47.053759 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:52:47.054381 (+ 622us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 02:52:47.054393 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0811 02:52:47.056535 (+ 2142us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.12.138.254:34683: Server oryx/127.12.138.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":239,"thread_start_us":120,"threads_started":1}
W20260811 02:52:47.056846 23683 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 02:52:47.050497 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.138.194:56465 (local address 127.12.138.254:34683)
0811 02:52:47.050661 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0811 02:52:47.050667 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 02:52:47.050957 (+ 290us) server_negotiation.cc:407] Connection header received
0811 02:52:47.051014 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:52:47.051016 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:52:47.051071 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:52:47.051160 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:52:47.051728 (+ 568us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:52:47.052291 (+ 563us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:52:47.053248 (+ 957us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:52:47.053511 (+ 263us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:52:47.053683 (+ 172us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:52:47.056742 (+ 3059us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.12.138.194:56465: BlockingRecv error: recv got EOF from 127.12.138.194:56465 (error 108)
Metrics: {"server-negotiator.queue_time_us":59}
Aug 11 02:52:47 dist-test-slave-9gcw krb5kdc[22473](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.138.195@KRBTEST.COM for oryx/127.12.138.254@KRBTEST.COM, Server not found in Kerberos database
Aug 11 02:52:47 dist-test-slave-9gcw krb5kdc[22473](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.138.195@KRBTEST.COM for oryx/127.12.138.254@KRBTEST.COM, Server not found in Kerberos database
I20260811 02:52:47.241570 23685 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:52:47.235700 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.138.254:34683 (local address 127.12.138.195:42413)
0811 02:52:47.236013 (+ 313us) negotiation.cc:107] Waiting for socket to connect
0811 02:52:47.236027 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 02:52:47.236134 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:52:47.236386 (+ 252us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:52:47.236388 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:52:47.236599 (+ 211us) client_negotiation.cc:189] Negotiated authn=SASL
0811 02:52:47.236695 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:52:47.236701 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:52:47.237546 (+ 845us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:52:47.237549 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:52:47.238129 (+ 580us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:52:47.238136 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:52:47.238564 (+ 428us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:52:47.238566 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:52:47.238627 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:52:47.239185 (+ 558us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 02:52:47.239199 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0811 02:52:47.241444 (+ 2245us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.12.138.254:34683: Server oryx/127.12.138.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":238,"thread_start_us":133,"threads_started":1}
W20260811 02:52:47.241730 23683 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 02:52:47.235772 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.138.195:42413 (local address 127.12.138.254:34683)
0811 02:52:47.235972 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0811 02:52:47.235979 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 02:52:47.236140 (+ 161us) server_negotiation.cc:407] Connection header received
0811 02:52:47.236210 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:52:47.236212 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:52:47.236264 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:52:47.236345 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:52:47.236837 (+ 492us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:52:47.237451 (+ 614us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:52:47.238238 (+ 787us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:52:47.238463 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:52:47.238547 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:52:47.241589 (+ 3042us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.12.138.195:42413: BlockingRecv error: recv got EOF from 127.12.138.195:42413 (error 108)
Metrics: {"server-negotiator.queue_time_us":95}
Aug 11 02:52:47 dist-test-slave-9gcw krb5kdc[22473](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.138.193@KRBTEST.COM for oryx/127.12.138.254@KRBTEST.COM, Server not found in Kerberos database
Aug 11 02:52:47 dist-test-slave-9gcw krb5kdc[22473](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.138.193@KRBTEST.COM for oryx/127.12.138.254@KRBTEST.COM, Server not found in Kerberos database
I20260811 02:52:47.834839 23686 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:52:47.828763 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.138.254:34683 (local address 127.12.138.193:34943)
0811 02:52:47.829058 (+ 295us) negotiation.cc:107] Waiting for socket to connect
0811 02:52:47.829081 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0811 02:52:47.829296 (+ 215us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:52:47.829646 (+ 350us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:52:47.829652 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:52:47.829914 (+ 262us) client_negotiation.cc:189] Negotiated authn=SASL
0811 02:52:47.830025 (+ 111us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:52:47.830031 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:52:47.830788 (+ 757us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:52:47.830791 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:52:47.831373 (+ 582us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:52:47.831381 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:52:47.831801 (+ 420us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:52:47.831803 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:52:47.831863 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:52:47.832396 (+ 533us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 02:52:47.832412 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0811 02:52:47.834700 (+ 2288us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.12.138.254:34683: Server oryx/127.12.138.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":198,"thread_start_us":100,"threads_started":1}
W20260811 02:52:47.834995 23687 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 02:52:47.828822 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.138.193:34943 (local address 127.12.138.254:34683)
0811 02:52:47.829103 (+ 281us) server_negotiation.cc:206] Beginning negotiation
0811 02:52:47.829108 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 02:52:47.829208 (+ 100us) server_negotiation.cc:407] Connection header received
0811 02:52:47.829421 (+ 213us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:52:47.829424 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:52:47.829487 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:52:47.829584 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:52:47.830141 (+ 557us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:52:47.830682 (+ 541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:52:47.831482 (+ 800us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:52:47.831706 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:52:47.831782 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:52:47.834853 (+ 3071us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.12.138.193:34943: BlockingRecv error: recv got EOF from 127.12.138.193:34943 (error 108)
Metrics: {"server-negotiator.queue_time_us":204,"thread_start_us":98,"threads_started":1}
Aug 11 02:52:48 dist-test-slave-9gcw krb5kdc[22473](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.138.194@KRBTEST.COM for oryx/127.12.138.254@KRBTEST.COM, Server not found in Kerberos database
Aug 11 02:52:48 dist-test-slave-9gcw krb5kdc[22473](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.138.194@KRBTEST.COM for oryx/127.12.138.254@KRBTEST.COM, Server not found in Kerberos database
I20260811 02:52:48.064437 23688 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:52:48.058030 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.138.254:34683 (local address 127.12.138.194:55401)
0811 02:52:48.058422 (+ 392us) negotiation.cc:107] Waiting for socket to connect
0811 02:52:48.058436 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 02:52:48.058573 (+ 137us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:52:48.058982 (+ 409us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:52:48.058985 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:52:48.059241 (+ 256us) client_negotiation.cc:189] Negotiated authn=SASL
0811 02:52:48.059386 (+ 145us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:52:48.059399 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:52:48.060253 (+ 854us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:52:48.060255 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:52:48.061003 (+ 748us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:52:48.061014 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:52:48.061539 (+ 525us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:52:48.061542 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:52:48.061610 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:52:48.062191 (+ 581us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 02:52:48.062204 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0811 02:52:48.064319 (+ 2115us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.12.138.254:34683: Server oryx/127.12.138.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":326,"thread_start_us":127,"threads_started":1}
W20260811 02:52:48.064646 23687 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 02:52:48.058092 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.138.194:55401 (local address 127.12.138.254:34683)
0811 02:52:48.058270 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0811 02:52:48.058275 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 02:52:48.058525 (+ 250us) server_negotiation.cc:407] Connection header received
0811 02:52:48.058794 (+ 269us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:52:48.058798 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:52:48.058861 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:52:48.058959 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:52:48.059559 (+ 600us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:52:48.060125 (+ 566us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:52:48.061129 (+ 1004us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:52:48.061411 (+ 282us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:52:48.061547 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:52:48.064484 (+ 2937us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.12.138.194:55401: BlockingRecv error: recv got EOF from 127.12.138.194:55401 (error 108)
Metrics: {"server-negotiator.queue_time_us":64}
I20260811 02:52:48.127805 12843 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDawEwS/build/debug/bin/kudu with pid 23135
I20260811 02:52:48.142406 12843 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDawEwS/build/debug/bin/kudu with pid 23314
I20260811 02:52:48.150800 12843 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDawEwS/build/debug/bin/kudu with pid 23470
I20260811 02:52:48.159493 12843 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDawEwS/build/debug/bin/kudu with pid 23052
2026-08-11T02:52:48Z chronyd exiting
[ OK ] SecurityITest.TestMismatchingPrincipals (18312 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationInsecureCluster
2026-08-11T02:52:48Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T02:52:48Z Disabled control of system clock
I20260811 02:52:48.196991 12843 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDawEwS/build/debug/bin/kudu
/tmp/dist-test-taskDawEwS/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786416707705494-12843-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786416707705494-12843-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786416707705494-12843-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786416707705494-12843-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.138.254:40073
--webserver_interface=127.12.138.254
--webserver_port=0
--builtin_ntp_servers=127.12.138.212:45473
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.12.138.254:40073
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260811 02:52:48.313833 23693 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 02:52:48.314126 23693 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 02:52:48.314160 23693 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 02:52:48.318328 23693 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 02:52:48.318398 23693 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 02:52:48.318414 23693 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 02:52:48.318434 23693 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 02:52:48.318449 23693 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 02:52:48.325313 23693 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.138.212:45473
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786416707705494-12843-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786416707705494-12843-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.12.138.254:40073
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.138.254:40073
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786416707705494-12843-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.12.138.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23693
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786416707705494-12843-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 02:09:49 UTC on ccycloud.quasar-fsnqqc.root.comops.site
build id 4579570
I20260811 02:52:48.326998 23693 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 02:52:48.329053 23693 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 02:52:48.339632 23699 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:52:48.340276 23700 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:52:48.340467 23702 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:52:48.341799 23693 server_base.cc:1034] running on GCE node
I20260811 02:52:48.342520 23693 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 02:52:48.344053 23693 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 02:52:48.345232 23693 hybrid_clock.cc:648] HybridClock initialized: now 1786416768345197 us; error 58 us; skew 500 ppm
I20260811 02:52:48.347429 23693 webserver.cc:460] Webserver started at http://127.12.138.254:45011/ using document root <none> and password file <none>
I20260811 02:52:48.348091 23693 fs_manager.cc:359] Metadata directory not provided
I20260811 02:52:48.348189 23693 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 02:52:48.348428 23693 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 02:52:48.351459 23693 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786416707705494-12843-0/minicluster-data/master-0/data/instance:
uuid: "0ad3918c24514d4c8a76d67c9f21e17a"
format_stamp: "Formatted at 2026-08-11 02:52:48 on dist-test-slave-9gcw"
server_key: "fb8a0e3282ef01ab4927a6d34226b30d"
server_key_iv: "439259606cbf4ed01f190be312674ee3"
server_key_version: "encryptionkey@0"
I20260811 02:52:48.352262 23693 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786416707705494-12843-0/minicluster-data/master-0/wal/instance:
uuid: "0ad3918c24514d4c8a76d67c9f21e17a"
format_stamp: "Formatted at 2026-08-11 02:52:48 on dist-test-slave-9gcw"
server_key: "fb8a0e3282ef01ab4927a6d34226b30d"
server_key_iv: "439259606cbf4ed01f190be312674ee3"
server_key_version: "encryptionkey@0"
I20260811 02:52:48.357785 23693 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.002s
I20260811 02:52:48.361518 23710 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 02:52:48.363193 23693 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260811 02:52:48.363338 23693 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786416707705494-12843-0/minicluster-data/master-0/data,/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786416707705494-12843-0/minicluster-data/master-0/wal
uuid: "0ad3918c24514d4c8a76d67c9f21e17a"
format_stamp: "Formatted at 2026-08-11 02:52:48 on dist-test-slave-9gcw"
server_key: "fb8a0e3282ef01ab4927a6d34226b30d"
server_key_iv: "439259606cbf4ed01f190be312674ee3"
server_key_version: "encryptionkey@0"
I20260811 02:52:48.363497 23693 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786416707705494-12843-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786416707705494-12843-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786416707705494-12843-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 02:52:48.387306 23693 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 02:52:48.388106 23693 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 02:52:48.388314 23693 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 02:52:48.396332 23693 rpc_server.cc:307] RPC server started. Bound to: 127.12.138.254:40073
I20260811 02:52:48.396348 23773 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.138.254:40073 every 8 connection(s)
I20260811 02:52:48.397466 23693 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786416707705494-12843-0/minicluster-data/master-0/data/info.pb
I20260811 02:52:48.400758 23774 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 02:52:48.403688 12843 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDawEwS/build/debug/bin/kudu as pid 23693
I20260811 02:52:48.403815 12843 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786416707705494-12843-0/minicluster-data/master-0/wal/instance
I20260811 02:52:48.404157 12843 external_mini_cluster.cc:1428] Setting key d1a02418a8c52b81630d8cf9680c9927
I20260811 02:52:48.407151 23774 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0ad3918c24514d4c8a76d67c9f21e17a: Bootstrap starting.
I20260811 02:52:48.409391 23774 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0ad3918c24514d4c8a76d67c9f21e17a: Neither blocks nor log segments found. Creating new log.
I20260811 02:52:48.410152 23774 log.cc:824] T 00000000000000000000000000000000 P 0ad3918c24514d4c8a76d67c9f21e17a: Log is configured to *not* fsync() on all Append() calls
I20260811 02:52:48.411957 23774 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0ad3918c24514d4c8a76d67c9f21e17a: No bootstrap required, opened a new log
I20260811 02:52:48.414531 23778 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:52:48.405841 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43072 (local address 127.12.138.254:40073)
0811 02:52:48.406542 (+ 701us) server_negotiation.cc:206] Beginning negotiation
0811 02:52:48.406558 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0811 02:52:48.406595 (+ 37us) server_negotiation.cc:407] Connection header received
0811 02:52:48.407416 (+ 821us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:52:48.407436 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:52:48.407855 (+ 419us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:52:48.408228 (+ 373us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:52:48.408716 (+ 488us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:52:48.409934 (+ 1218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:52:48.410947 (+ 1013us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:52:48.411457 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:52:48.411662 (+ 205us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:52:48.412605 (+ 943us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:52:48.412611 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:52:48.412630 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 02:52:48.413029 (+ 399us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:52:48.413087 (+ 58us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 02:52:48.413108 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:52:48.413364 (+ 256us) server_negotiation.cc:1035] Waiting for connection context
0811 02:52:48.413525 (+ 161us) server_negotiation.cc:299] Negotiation successful
0811 02:52:48.413809 (+ 284us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":319,"thread_start_us":143,"threads_started":1}
I20260811 02:52:48.415740 23774 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0ad3918c24514d4c8a76d67c9f21e17a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0ad3918c24514d4c8a76d67c9f21e17a" member_type: VOTER last_known_addr { host: "127.12.138.254" port: 40073 } }
I20260811 02:52:48.416010 23774 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0ad3918c24514d4c8a76d67c9f21e17a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 02:52:48.416064 23774 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0ad3918c24514d4c8a76d67c9f21e17a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0ad3918c24514d4c8a76d67c9f21e17a, State: Initialized, Role: FOLLOWER
I20260811 02:52:48.416509 23774 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0ad3918c24514d4c8a76d67c9f21e17a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0ad3918c24514d4c8a76d67c9f21e17a" member_type: VOTER last_known_addr { host: "127.12.138.254" port: 40073 } }
I20260811 02:52:48.416677 23774 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0ad3918c24514d4c8a76d67c9f21e17a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 02:52:48.416749 23774 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0ad3918c24514d4c8a76d67c9f21e17a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 02:52:48.416852 23774 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0ad3918c24514d4c8a76d67c9f21e17a [term 0 FOLLOWER]: Advancing to term 1
I20260811 02:52:48.417819 23774 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0ad3918c24514d4c8a76d67c9f21e17a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0ad3918c24514d4c8a76d67c9f21e17a" member_type: VOTER last_known_addr { host: "127.12.138.254" port: 40073 } }
I20260811 02:52:48.418143 23774 leader_election.cc:302] T 00000000000000000000000000000000 P 0ad3918c24514d4c8a76d67c9f21e17a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0ad3918c24514d4c8a76d67c9f21e17a; no voters:
I20260811 02:52:48.418481 23774 leader_election.cc:288] T 00000000000000000000000000000000 P 0ad3918c24514d4c8a76d67c9f21e17a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 02:52:48.418602 23780 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0ad3918c24514d4c8a76d67c9f21e17a [term 1 FOLLOWER]: Leader election won for term 1
I20260811 02:52:48.418844 23780 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0ad3918c24514d4c8a76d67c9f21e17a [term 1 LEADER]: Becoming Leader. State: Replica: 0ad3918c24514d4c8a76d67c9f21e17a, State: Running, Role: LEADER
I20260811 02:52:48.419169 23780 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0ad3918c24514d4c8a76d67c9f21e17a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0ad3918c24514d4c8a76d67c9f21e17a" member_type: VOTER last_known_addr { host: "127.12.138.254" port: 40073 } }
I20260811 02:52:48.419732 23774 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0ad3918c24514d4c8a76d67c9f21e17a [sys.catalog]: configured and running, proceeding with master startup.
I20260811 02:52:48.420902 23781 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0ad3918c24514d4c8a76d67c9f21e17a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0ad3918c24514d4c8a76d67c9f21e17a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0ad3918c24514d4c8a76d67c9f21e17a" member_type: VOTER last_known_addr { host: "127.12.138.254" port: 40073 } } }
I20260811 02:52:48.421036 23781 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0ad3918c24514d4c8a76d67c9f21e17a [sys.catalog]: This master's current role is: LEADER
I20260811 02:52:48.421344 23782 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0ad3918c24514d4c8a76d67c9f21e17a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0ad3918c24514d4c8a76d67c9f21e17a. Latest consensus state: current_term: 1 leader_uuid: "0ad3918c24514d4c8a76d67c9f21e17a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0ad3918c24514d4c8a76d67c9f21e17a" member_type: VOTER last_known_addr { host: "127.12.138.254" port: 40073 } } }
I20260811 02:52:48.421455 23782 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0ad3918c24514d4c8a76d67c9f21e17a [sys.catalog]: This master's current role is: LEADER
I20260811 02:52:48.421554 23789 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 02:52:48.424467 23789 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 02:52:48.430784 23789 catalog_manager.cc:1347] Generated new cluster ID: 54d0080abef844139f3a156d86e9d875
I20260811 02:52:48.430873 23789 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 02:52:48.461467 23789 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 02:52:48.462805 23789 catalog_manager.cc:1504] Loading token signing keys...
I20260811 02:52:48.472625 23789 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0ad3918c24514d4c8a76d67c9f21e17a: Generated new TSK 0
I20260811 02:52:48.473590 23789 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 02:52:48.478098 12843 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDawEwS/build/debug/bin/kudu
/tmp/dist-test-taskDawEwS/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786416707705494-12843-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786416707705494-12843-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786416707705494-12843-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786416707705494-12843-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.138.193:0
--local_ip_for_outbound_sockets=127.12.138.193
--webserver_interface=127.12.138.193
--webserver_port=0
--tserver_master_addrs=127.12.138.254:40073
--builtin_ntp_servers=127.12.138.212:45473
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
W20260811 02:52:48.599262 23800 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 02:52:48.599532 23800 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 02:52:48.599570 23800 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 02:52:48.603812 23800 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 02:52:48.603881 23800 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 02:52:48.603936 23800 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.138.193
I20260811 02:52:48.608736 23800 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.138.212:45473
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786416707705494-12843-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786416707705494-12843-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.138.193:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786416707705494-12843-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.12.138.193
--webserver_port=0
--tserver_master_addrs=127.12.138.254:40073
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23800
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.138.193
--log_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786416707705494-12843-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 02:09:49 UTC on ccycloud.quasar-fsnqqc.root.comops.site
build id 4579570
I20260811 02:52:48.609922 23800 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 02:52:48.611421 23800 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 02:52:48.622015 23809 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:52:48.621986 23807 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:52:48.621969 23811 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:52:48.622468 23800 server_base.cc:1034] running on GCE node
I20260811 02:52:48.622852 23800 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 02:52:48.623450 23800 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 02:52:48.624609 23800 hybrid_clock.cc:648] HybridClock initialized: now 1786416768624592 us; error 35 us; skew 500 ppm
I20260811 02:52:48.626694 23800 webserver.cc:460] Webserver started at http://127.12.138.193:38607/ using document root <none> and password file <none>
I20260811 02:52:48.627300 23800 fs_manager.cc:359] Metadata directory not provided
I20260811 02:52:48.627364 23800 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 02:52:48.627591 23800 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 02:52:48.629774 23800 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786416707705494-12843-0/minicluster-data/ts-0/data/instance:
uuid: "ac53c7b6f9574905805f6935ea4e2977"
format_stamp: "Formatted at 2026-08-11 02:52:48 on dist-test-slave-9gcw"
server_key: "f4b3fd115c8a084f34547d0065a55071"
server_key_iv: "38d0269f19032de804c86754906793d6"
server_key_version: "encryptionkey@0"
I20260811 02:52:48.630365 23800 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786416707705494-12843-0/minicluster-data/ts-0/wal/instance:
uuid: "ac53c7b6f9574905805f6935ea4e2977"
format_stamp: "Formatted at 2026-08-11 02:52:48 on dist-test-slave-9gcw"
server_key: "f4b3fd115c8a084f34547d0065a55071"
server_key_iv: "38d0269f19032de804c86754906793d6"
server_key_version: "encryptionkey@0"
I20260811 02:52:48.634405 23800 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260811 02:52:48.637106 23818 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 02:52:48.638536 23800 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 02:52:48.638674 23800 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786416707705494-12843-0/minicluster-data/ts-0/data,/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786416707705494-12843-0/minicluster-data/ts-0/wal
uuid: "ac53c7b6f9574905805f6935ea4e2977"
format_stamp: "Formatted at 2026-08-11 02:52:48 on dist-test-slave-9gcw"
server_key: "f4b3fd115c8a084f34547d0065a55071"
server_key_iv: "38d0269f19032de804c86754906793d6"
server_key_version: "encryptionkey@0"
I20260811 02:52:48.638795 23800 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786416707705494-12843-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786416707705494-12843-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786416707705494-12843-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 02:52:48.669087 23800 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 02:52:48.670019 23800 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 02:52:48.670225 23800 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 02:52:48.670847 23800 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 02:52:48.671855 23800 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 02:52:48.671929 23800 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 02:52:48.672004 23800 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 02:52:48.672046 23800 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 02:52:48.682050 23800 rpc_server.cc:307] RPC server started. Bound to: 127.12.138.193:45881
I20260811 02:52:48.682078 23945 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.138.193:45881 every 8 connection(s)
I20260811 02:52:48.683094 23800 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786416707705494-12843-0/minicluster-data/ts-0/data/info.pb
I20260811 02:52:48.686690 12843 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDawEwS/build/debug/bin/kudu as pid 23800
I20260811 02:52:48.686816 12843 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786416707705494-12843-0/minicluster-data/ts-0/wal/instance
I20260811 02:52:48.687216 12843 external_mini_cluster.cc:1428] Setting key de99d73b76a022651e7e572a4f8f7a5b
I20260811 02:52:48.689442 12843 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDawEwS/build/debug/bin/kudu
/tmp/dist-test-taskDawEwS/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786416707705494-12843-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786416707705494-12843-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786416707705494-12843-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786416707705494-12843-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.138.194:0
--local_ip_for_outbound_sockets=127.12.138.194
--webserver_interface=127.12.138.194
--webserver_port=0
--tserver_master_addrs=127.12.138.254:40073
--builtin_ntp_servers=127.12.138.212:45473
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260811 02:52:48.694657 23778 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:52:48.685189 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.138.193:60655 (local address 127.12.138.254:40073)
0811 02:52:48.685324 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0811 02:52:48.685327 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 02:52:48.686461 (+ 1134us) server_negotiation.cc:407] Connection header received
0811 02:52:48.688564 (+ 2103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:52:48.688568 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:52:48.688629 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:52:48.688728 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:52:48.689872 (+ 1144us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:52:48.690482 (+ 610us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:52:48.692349 (+ 1867us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:52:48.692572 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:52:48.692660 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:52:48.694118 (+ 1458us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:52:48.694122 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:52:48.694124 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 02:52:48.694137 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:52:48.694157 (+ 20us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 02:52:48.694165 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:52:48.694245 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0811 02:52:48.694439 (+ 194us) server_negotiation.cc:299] Negotiation successful
0811 02:52:48.694509 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260811 02:52:48.695577 23948 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:52:48.685508 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.138.254:40073 (local address 127.12.138.193:60655)
0811 02:52:48.686261 (+ 753us) negotiation.cc:107] Waiting for socket to connect
0811 02:52:48.686304 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0811 02:52:48.688320 (+ 2016us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:52:48.688931 (+ 611us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:52:48.688938 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:52:48.689028 (+ 90us) client_negotiation.cc:189] Negotiated authn=SASL
0811 02:52:48.689712 (+ 684us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:52:48.689730 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:52:48.690639 (+ 909us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:52:48.690647 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:52:48.692196 (+ 1549us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:52:48.692210 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:52:48.692752 (+ 542us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:52:48.692755 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:52:48.692954 (+ 199us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:52:48.693744 (+ 790us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 02:52:48.693746 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0811 02:52:48.693805 (+ 59us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 02:52:48.693901 (+ 96us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 02:52:48.694261 (+ 360us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 02:52:48.694272 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 02:52:48.694284 (+ 12us) client_negotiation.cc:769] Sending connection context
0811 02:52:48.694421 (+ 137us) client_negotiation.cc:240] Negotiation successful
0811 02:52:48.694734 (+ 313us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":315,"thread_start_us":119,"threads_started":1}
I20260811 02:52:48.696916 23946 heartbeater.cc:347] Connected to a master server at 127.12.138.254:40073
I20260811 02:52:48.697256 23946 heartbeater.cc:464] Registering TS with master...
I20260811 02:52:48.697817 23946 heartbeater.cc:511] Master 127.12.138.254:40073 requested a full tablet report, sending...
I20260811 02:52:48.699600 23735 ts_manager.cc:194] Registered new tserver with Master: ac53c7b6f9574905805f6935ea4e2977 (127.12.138.193:45881)
I20260811 02:52:48.701473 23735 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.12.138.193:60655
W20260811 02:52:48.809089 23949 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 02:52:48.809373 23949 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 02:52:48.809412 23949 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 02:52:48.813347 23949 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 02:52:48.813405 23949 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 02:52:48.813457 23949 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.138.194
I20260811 02:52:48.818288 23949 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.138.212:45473
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786416707705494-12843-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786416707705494-12843-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.138.194:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786416707705494-12843-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.12.138.194
--webserver_port=0
--tserver_master_addrs=127.12.138.254:40073
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23949
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.138.194
--log_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786416707705494-12843-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 02:09:49 UTC on ccycloud.quasar-fsnqqc.root.comops.site
build id 4579570
I20260811 02:52:48.819360 23949 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 02:52:48.820919 23949 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 02:52:48.831099 23955 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:52:48.831118 23956 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:52:48.831117 23958 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:52:48.831394 23949 server_base.cc:1034] running on GCE node
I20260811 02:52:48.831821 23949 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 02:52:48.832418 23949 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 02:52:48.833608 23949 hybrid_clock.cc:648] HybridClock initialized: now 1786416768833588 us; error 33 us; skew 500 ppm
I20260811 02:52:48.835564 23949 webserver.cc:460] Webserver started at http://127.12.138.194:41691/ using document root <none> and password file <none>
I20260811 02:52:48.836179 23949 fs_manager.cc:359] Metadata directory not provided
I20260811 02:52:48.836280 23949 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 02:52:48.836527 23949 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 02:52:48.838694 23949 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786416707705494-12843-0/minicluster-data/ts-1/data/instance:
uuid: "dadd96d980b749f093fc87869275fe94"
format_stamp: "Formatted at 2026-08-11 02:52:48 on dist-test-slave-9gcw"
server_key: "d07a11a83ebd7de7e874d15f78e65e8e"
server_key_iv: "155f31edbf0c9078d4a9faaed9d2ef21"
server_key_version: "encryptionkey@0"
I20260811 02:52:48.839277 23949 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786416707705494-12843-0/minicluster-data/ts-1/wal/instance:
uuid: "dadd96d980b749f093fc87869275fe94"
format_stamp: "Formatted at 2026-08-11 02:52:48 on dist-test-slave-9gcw"
server_key: "d07a11a83ebd7de7e874d15f78e65e8e"
server_key_iv: "155f31edbf0c9078d4a9faaed9d2ef21"
server_key_version: "encryptionkey@0"
I20260811 02:52:48.843288 23949 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260811 02:52:48.846287 23965 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 02:52:48.847970 23949 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 02:52:48.848109 23949 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786416707705494-12843-0/minicluster-data/ts-1/data,/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786416707705494-12843-0/minicluster-data/ts-1/wal
uuid: "dadd96d980b749f093fc87869275fe94"
format_stamp: "Formatted at 2026-08-11 02:52:48 on dist-test-slave-9gcw"
server_key: "d07a11a83ebd7de7e874d15f78e65e8e"
server_key_iv: "155f31edbf0c9078d4a9faaed9d2ef21"
server_key_version: "encryptionkey@0"
I20260811 02:52:48.848255 23949 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786416707705494-12843-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786416707705494-12843-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786416707705494-12843-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 02:52:48.872810 23949 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 02:52:48.873768 23949 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 02:52:48.874038 23949 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 02:52:48.874734 23949 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 02:52:48.875895 23949 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 02:52:48.875980 23949 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 02:52:48.876062 23949 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 02:52:48.876112 23949 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 02:52:48.886994 23949 rpc_server.cc:307] RPC server started. Bound to: 127.12.138.194:41313
I20260811 02:52:48.887017 24102 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.138.194:41313 every 8 connection(s)
I20260811 02:52:48.888185 23949 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786416707705494-12843-0/minicluster-data/ts-1/data/info.pb
I20260811 02:52:48.898471 12843 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDawEwS/build/debug/bin/kudu as pid 23949
I20260811 02:52:48.898628 12843 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786416707705494-12843-0/minicluster-data/ts-1/wal/instance
I20260811 02:52:48.899039 12843 external_mini_cluster.cc:1428] Setting key fa503b82149757cdc25efb7552cc74a4
I20260811 02:52:48.899302 23778 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:52:48.890459 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.138.194:38457 (local address 127.12.138.254:40073)
0811 02:52:48.890624 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0811 02:52:48.890628 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 02:52:48.891941 (+ 1313us) server_negotiation.cc:407] Connection header received
0811 02:52:48.893592 (+ 1651us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:52:48.893595 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:52:48.893664 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:52:48.893792 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:52:48.894877 (+ 1085us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:52:48.895499 (+ 622us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:52:48.896958 (+ 1459us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:52:48.897257 (+ 299us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:52:48.897384 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:52:48.898683 (+ 1299us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:52:48.898689 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:52:48.898694 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 02:52:48.898712 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:52:48.898741 (+ 29us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 02:52:48.898750 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:52:48.898823 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0811 02:52:48.899043 (+ 220us) server_negotiation.cc:299] Negotiation successful
0811 02:52:48.899128 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260811 02:52:48.900377 24105 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:52:48.890950 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.138.254:40073 (local address 127.12.138.194:38457)
0811 02:52:48.891738 (+ 788us) negotiation.cc:107] Waiting for socket to connect
0811 02:52:48.891787 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0811 02:52:48.893328 (+ 1541us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:52:48.894003 (+ 675us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:52:48.894011 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:52:48.894110 (+ 99us) client_negotiation.cc:189] Negotiated authn=SASL
0811 02:52:48.894694 (+ 584us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:52:48.894710 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:52:48.895670 (+ 960us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:52:48.895675 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:52:48.896692 (+ 1017us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:52:48.896702 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:52:48.897438 (+ 736us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:52:48.897442 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:52:48.897629 (+ 187us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:52:48.898292 (+ 663us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 02:52:48.898295 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0811 02:52:48.898353 (+ 58us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 02:52:48.898444 (+ 91us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 02:52:48.898850 (+ 406us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 02:52:48.898862 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 02:52:48.898870 (+ 8us) client_negotiation.cc:769] Sending connection context
0811 02:52:48.899029 (+ 159us) client_negotiation.cc:240] Negotiation successful
0811 02:52:48.899356 (+ 327us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":304,"thread_start_us":131,"threads_started":1}
I20260811 02:52:48.901316 12843 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDawEwS/build/debug/bin/kudu
/tmp/dist-test-taskDawEwS/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786416707705494-12843-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786416707705494-12843-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786416707705494-12843-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786416707705494-12843-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.138.195:0
--local_ip_for_outbound_sockets=127.12.138.195
--webserver_interface=127.12.138.195
--webserver_port=0
--tserver_master_addrs=127.12.138.254:40073
--builtin_ntp_servers=127.12.138.212:45473
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260811 02:52:48.901612 24103 heartbeater.cc:347] Connected to a master server at 127.12.138.254:40073
I20260811 02:52:48.901889 24103 heartbeater.cc:464] Registering TS with master...
I20260811 02:52:48.902434 24103 heartbeater.cc:511] Master 127.12.138.254:40073 requested a full tablet report, sending...
I20260811 02:52:48.903633 23735 ts_manager.cc:194] Registered new tserver with Master: dadd96d980b749f093fc87869275fe94 (127.12.138.194:41313)
I20260811 02:52:48.904289 23735 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.12.138.194:38457
W20260811 02:52:49.022681 24107 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 02:52:49.022955 24107 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 02:52:49.022994 24107 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 02:52:49.027057 24107 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 02:52:49.027151 24107 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 02:52:49.027242 24107 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.138.195
I20260811 02:52:49.032284 24107 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.138.212:45473
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786416707705494-12843-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786416707705494-12843-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.138.195:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786416707705494-12843-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.12.138.195
--webserver_port=0
--tserver_master_addrs=127.12.138.254:40073
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24107
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.138.195
--log_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786416707705494-12843-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 02:09:49 UTC on ccycloud.quasar-fsnqqc.root.comops.site
build id 4579570
I20260811 02:52:49.033478 24107 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 02:52:49.034907 24107 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 02:52:49.045740 24118 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:52:49.045737 24116 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:52:49.045782 24115 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:52:49.046119 24107 server_base.cc:1034] running on GCE node
I20260811 02:52:49.046602 24107 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 02:52:49.047295 24107 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 02:52:49.048511 24107 hybrid_clock.cc:648] HybridClock initialized: now 1786416769048498 us; error 36 us; skew 500 ppm
I20260811 02:52:49.050727 24107 webserver.cc:460] Webserver started at http://127.12.138.195:41481/ using document root <none> and password file <none>
I20260811 02:52:49.051399 24107 fs_manager.cc:359] Metadata directory not provided
I20260811 02:52:49.051527 24107 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 02:52:49.051934 24107 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 02:52:49.054539 24107 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786416707705494-12843-0/minicluster-data/ts-2/data/instance:
uuid: "a03a0e3562d4485d97147fa344ca82ac"
format_stamp: "Formatted at 2026-08-11 02:52:49 on dist-test-slave-9gcw"
server_key: "9fc963507baf7ede7917d8e5d3082225"
server_key_iv: "11ccfe1e163e5253dcee3c06536c8c5e"
server_key_version: "encryptionkey@0"
I20260811 02:52:49.055387 24107 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786416707705494-12843-0/minicluster-data/ts-2/wal/instance:
uuid: "a03a0e3562d4485d97147fa344ca82ac"
format_stamp: "Formatted at 2026-08-11 02:52:49 on dist-test-slave-9gcw"
server_key: "9fc963507baf7ede7917d8e5d3082225"
server_key_iv: "11ccfe1e163e5253dcee3c06536c8c5e"
server_key_version: "encryptionkey@0"
I20260811 02:52:49.061496 24107 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.003s sys 0.005s
I20260811 02:52:49.065428 24125 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 02:52:49.066934 24107 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260811 02:52:49.067037 24107 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786416707705494-12843-0/minicluster-data/ts-2/data,/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786416707705494-12843-0/minicluster-data/ts-2/wal
uuid: "a03a0e3562d4485d97147fa344ca82ac"
format_stamp: "Formatted at 2026-08-11 02:52:49 on dist-test-slave-9gcw"
server_key: "9fc963507baf7ede7917d8e5d3082225"
server_key_iv: "11ccfe1e163e5253dcee3c06536c8c5e"
server_key_version: "encryptionkey@0"
I20260811 02:52:49.067188 24107 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786416707705494-12843-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786416707705494-12843-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786416707705494-12843-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 02:52:49.078045 24107 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 02:52:49.078698 24107 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 02:52:49.078898 24107 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 02:52:49.079553 24107 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 02:52:49.080510 24107 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 02:52:49.080569 24107 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 02:52:49.080632 24107 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 02:52:49.080694 24107 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 02:52:49.089857 24107 rpc_server.cc:307] RPC server started. Bound to: 127.12.138.195:41505
I20260811 02:52:49.089881 24257 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.138.195:41505 every 8 connection(s)
I20260811 02:52:49.090778 24107 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786416707705494-12843-0/minicluster-data/ts-2/data/info.pb
I20260811 02:52:49.099838 12843 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDawEwS/build/debug/bin/kudu as pid 24107
I20260811 02:52:49.099974 12843 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786416707705494-12843-0/minicluster-data/ts-2/wal/instance
I20260811 02:52:49.100363 12843 external_mini_cluster.cc:1428] Setting key b5e3497a518554f4533df2cff922080f
I20260811 02:52:49.100404 23778 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:52:49.092694 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.138.195:47751 (local address 127.12.138.254:40073)
0811 02:52:49.092838 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0811 02:52:49.092841 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 02:52:49.094024 (+ 1183us) server_negotiation.cc:407] Connection header received
0811 02:52:49.095504 (+ 1480us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:52:49.095508 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:52:49.095570 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:52:49.095671 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:52:49.096576 (+ 905us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:52:49.097115 (+ 539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:52:49.098363 (+ 1248us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:52:49.098587 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:52:49.098695 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:52:49.099830 (+ 1135us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:52:49.099836 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:52:49.099842 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 02:52:49.099855 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:52:49.099873 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 02:52:49.099881 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:52:49.099974 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0811 02:52:49.100231 (+ 257us) server_negotiation.cc:299] Negotiation successful
0811 02:52:49.100286 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260811 02:52:49.101684 24260 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:52:49.093121 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.138.254:40073 (local address 127.12.138.195:47751)
0811 02:52:49.093860 (+ 739us) negotiation.cc:107] Waiting for socket to connect
0811 02:52:49.093907 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0811 02:52:49.095249 (+ 1342us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:52:49.095849 (+ 600us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:52:49.095856 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:52:49.095939 (+ 83us) client_negotiation.cc:189] Negotiated authn=SASL
0811 02:52:49.096436 (+ 497us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:52:49.096449 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:52:49.097331 (+ 882us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:52:49.097334 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:52:49.098200 (+ 866us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:52:49.098208 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:52:49.098747 (+ 539us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:52:49.098752 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:52:49.098922 (+ 170us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:52:49.099562 (+ 640us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 02:52:49.099564 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0811 02:52:49.099604 (+ 40us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 02:52:49.099666 (+ 62us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 02:52:49.100032 (+ 366us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 02:52:49.100045 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 02:52:49.100052 (+ 7us) client_negotiation.cc:769] Sending connection context
0811 02:52:49.100236 (+ 184us) client_negotiation.cc:240] Negotiation successful
0811 02:52:49.100553 (+ 317us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":283,"thread_start_us":127,"threads_started":1}
I20260811 02:52:49.102890 24258 heartbeater.cc:347] Connected to a master server at 127.12.138.254:40073
I20260811 02:52:49.103101 24258 heartbeater.cc:464] Registering TS with master...
I20260811 02:52:49.103595 24258 heartbeater.cc:511] Master 127.12.138.254:40073 requested a full tablet report, sending...
I20260811 02:52:49.104599 23735 ts_manager.cc:194] Registered new tserver with Master: a03a0e3562d4485d97147fa344ca82ac (127.12.138.195:41505)
I20260811 02:52:49.105311 23735 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.12.138.195:47751
I20260811 02:52:49.114880 12843 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260811 02:52:49.118050 23778 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 02:52:49.117082 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43080 (local address 127.12.138.254:40073)
0811 02:52:49.117286 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0811 02:52:49.117289 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 02:52:49.117397 (+ 108us) server_negotiation.cc:407] Connection header received
0811 02:52:49.117531 (+ 134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:52:49.117534 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:52:49.117574 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:52:49.117643 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:52:49.117903 (+ 260us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43080: BlockingRecv error: recv got EOF from 127.0.0.1:43080 (error 108)
Metrics: {"server-negotiator.queue_time_us":100}
I20260811 02:52:49.118207 12843 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDawEwS/build/debug/bin/kudu with pid 23800
I20260811 02:52:49.130856 12843 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDawEwS/build/debug/bin/kudu with pid 23949
I20260811 02:52:49.142122 12843 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDawEwS/build/debug/bin/kudu with pid 24107
I20260811 02:52:49.152912 12843 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDawEwS/build/debug/bin/kudu with pid 23693
2026-08-11T02:52:49Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationInsecureCluster (992 ms)
[ RUN ] SecurityITest.TestRequireEncryptionInsecureCluster
2026-08-11T02:52:49Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T02:52:49Z Disabled control of system clock
I20260811 02:52:49.190277 12843 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDawEwS/build/debug/bin/kudu
/tmp/dist-test-taskDawEwS/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786416707705494-12843-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786416707705494-12843-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786416707705494-12843-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786416707705494-12843-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.138.254:42005
--webserver_interface=127.12.138.254
--webserver_port=0
--builtin_ntp_servers=127.12.138.212:35887
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.12.138.254:42005
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260811 02:52:49.310786 24269 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 02:52:49.311058 24269 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 02:52:49.311096 24269 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 02:52:49.314986 24269 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 02:52:49.315037 24269 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 02:52:49.315052 24269 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 02:52:49.315068 24269 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 02:52:49.315083 24269 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 02:52:49.319823 24269 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.138.212:35887
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786416707705494-12843-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786416707705494-12843-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.12.138.254:42005
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.138.254:42005
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786416707705494-12843-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.12.138.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24269
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786416707705494-12843-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 02:09:49 UTC on ccycloud.quasar-fsnqqc.root.comops.site
build id 4579570
I20260811 02:52:49.321110 24269 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 02:52:49.322741 24269 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 02:52:49.331549 24278 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:52:49.331537 24275 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:52:49.331748 24269 server_base.cc:1034] running on GCE node
W20260811 02:52:49.331517 24276 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:52:49.332428 24269 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 02:52:49.333683 24269 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 02:52:49.334892 24269 hybrid_clock.cc:648] HybridClock initialized: now 1786416769334871 us; error 30 us; skew 500 ppm
I20260811 02:52:49.337066 24269 webserver.cc:460] Webserver started at http://127.12.138.254:46589/ using document root <none> and password file <none>
I20260811 02:52:49.337725 24269 fs_manager.cc:359] Metadata directory not provided
I20260811 02:52:49.337826 24269 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 02:52:49.338079 24269 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 02:52:49.340277 24269 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786416707705494-12843-0/minicluster-data/master-0/data/instance:
uuid: "7c645129730f4b278d785393d40feb1f"
format_stamp: "Formatted at 2026-08-11 02:52:49 on dist-test-slave-9gcw"
server_key: "ebe8260c78c375ba4ec55d2c466c936b"
server_key_iv: "869dff0dd6f29092d8fd4a1440b5c021"
server_key_version: "encryptionkey@0"
I20260811 02:52:49.340907 24269 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786416707705494-12843-0/minicluster-data/master-0/wal/instance:
uuid: "7c645129730f4b278d785393d40feb1f"
format_stamp: "Formatted at 2026-08-11 02:52:49 on dist-test-slave-9gcw"
server_key: "ebe8260c78c375ba4ec55d2c466c936b"
server_key_iv: "869dff0dd6f29092d8fd4a1440b5c021"
server_key_version: "encryptionkey@0"
I20260811 02:52:49.344964 24269 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.000s
I20260811 02:52:49.347608 24285 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 02:52:49.348971 24269 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260811 02:52:49.349184 24269 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786416707705494-12843-0/minicluster-data/master-0/data,/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786416707705494-12843-0/minicluster-data/master-0/wal
uuid: "7c645129730f4b278d785393d40feb1f"
format_stamp: "Formatted at 2026-08-11 02:52:49 on dist-test-slave-9gcw"
server_key: "ebe8260c78c375ba4ec55d2c466c936b"
server_key_iv: "869dff0dd6f29092d8fd4a1440b5c021"
server_key_version: "encryptionkey@0"
I20260811 02:52:49.349336 24269 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786416707705494-12843-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786416707705494-12843-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786416707705494-12843-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 02:52:49.354316 24269 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 02:52:49.354993 24269 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 02:52:49.355217 24269 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 02:52:49.363226 24269 rpc_server.cc:307] RPC server started. Bound to: 127.12.138.254:42005
I20260811 02:52:49.363229 24348 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.138.254:42005 every 8 connection(s)
I20260811 02:52:49.364248 24269 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786416707705494-12843-0/minicluster-data/master-0/data/info.pb
I20260811 02:52:49.367239 12843 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDawEwS/build/debug/bin/kudu as pid 24269
I20260811 02:52:49.367399 12843 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786416707705494-12843-0/minicluster-data/master-0/wal/instance
I20260811 02:52:49.367736 12843 external_mini_cluster.cc:1428] Setting key c1c20c2652e95f9064ef77066c46b941
I20260811 02:52:49.367767 24349 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 02:52:49.373558 24352 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:52:49.369081 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43480 (local address 127.12.138.254:42005)
0811 02:52:49.369813 (+ 732us) server_negotiation.cc:206] Beginning negotiation
0811 02:52:49.369827 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0811 02:52:49.369862 (+ 35us) server_negotiation.cc:407] Connection header received
0811 02:52:49.370540 (+ 678us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:52:49.370558 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:52:49.370980 (+ 422us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:52:49.371351 (+ 371us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:52:49.372208 (+ 857us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:52:49.372213 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:52:49.372225 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 02:52:49.372235 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:52:49.372291 (+ 56us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 02:52:49.372308 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:52:49.372467 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0811 02:52:49.372540 (+ 73us) server_negotiation.cc:299] Negotiation successful
0811 02:52:49.372788 (+ 248us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":365,"thread_start_us":155,"threads_started":1}
I20260811 02:52:49.375082 24349 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7c645129730f4b278d785393d40feb1f: Bootstrap starting.
I20260811 02:52:49.377455 24349 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7c645129730f4b278d785393d40feb1f: Neither blocks nor log segments found. Creating new log.
I20260811 02:52:49.378269 24349 log.cc:824] T 00000000000000000000000000000000 P 7c645129730f4b278d785393d40feb1f: Log is configured to *not* fsync() on all Append() calls
I20260811 02:52:49.380599 24349 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7c645129730f4b278d785393d40feb1f: No bootstrap required, opened a new log
I20260811 02:52:49.384164 24349 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7c645129730f4b278d785393d40feb1f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7c645129730f4b278d785393d40feb1f" member_type: VOTER last_known_addr { host: "127.12.138.254" port: 42005 } }
I20260811 02:52:49.384406 24349 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7c645129730f4b278d785393d40feb1f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 02:52:49.384446 24349 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7c645129730f4b278d785393d40feb1f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7c645129730f4b278d785393d40feb1f, State: Initialized, Role: FOLLOWER
I20260811 02:52:49.384822 24349 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7c645129730f4b278d785393d40feb1f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7c645129730f4b278d785393d40feb1f" member_type: VOTER last_known_addr { host: "127.12.138.254" port: 42005 } }
I20260811 02:52:49.384938 24349 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7c645129730f4b278d785393d40feb1f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 02:52:49.384979 24349 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7c645129730f4b278d785393d40feb1f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 02:52:49.385051 24349 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7c645129730f4b278d785393d40feb1f [term 0 FOLLOWER]: Advancing to term 1
I20260811 02:52:49.386128 24349 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7c645129730f4b278d785393d40feb1f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7c645129730f4b278d785393d40feb1f" member_type: VOTER last_known_addr { host: "127.12.138.254" port: 42005 } }
I20260811 02:52:49.386425 24349 leader_election.cc:302] T 00000000000000000000000000000000 P 7c645129730f4b278d785393d40feb1f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7c645129730f4b278d785393d40feb1f; no voters:
I20260811 02:52:49.386809 24349 leader_election.cc:288] T 00000000000000000000000000000000 P 7c645129730f4b278d785393d40feb1f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 02:52:49.386924 24354 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7c645129730f4b278d785393d40feb1f [term 1 FOLLOWER]: Leader election won for term 1
I20260811 02:52:49.387112 24354 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7c645129730f4b278d785393d40feb1f [term 1 LEADER]: Becoming Leader. State: Replica: 7c645129730f4b278d785393d40feb1f, State: Running, Role: LEADER
I20260811 02:52:49.387413 24354 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7c645129730f4b278d785393d40feb1f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7c645129730f4b278d785393d40feb1f" member_type: VOTER last_known_addr { host: "127.12.138.254" port: 42005 } }
I20260811 02:52:49.388150 24349 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7c645129730f4b278d785393d40feb1f [sys.catalog]: configured and running, proceeding with master startup.
I20260811 02:52:49.389230 24355 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7c645129730f4b278d785393d40feb1f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7c645129730f4b278d785393d40feb1f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7c645129730f4b278d785393d40feb1f" member_type: VOTER last_known_addr { host: "127.12.138.254" port: 42005 } } }
I20260811 02:52:49.389276 24356 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7c645129730f4b278d785393d40feb1f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7c645129730f4b278d785393d40feb1f. Latest consensus state: current_term: 1 leader_uuid: "7c645129730f4b278d785393d40feb1f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7c645129730f4b278d785393d40feb1f" member_type: VOTER last_known_addr { host: "127.12.138.254" port: 42005 } } }
I20260811 02:52:49.389371 24355 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7c645129730f4b278d785393d40feb1f [sys.catalog]: This master's current role is: LEADER
I20260811 02:52:49.389405 24356 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7c645129730f4b278d785393d40feb1f [sys.catalog]: This master's current role is: LEADER
I20260811 02:52:49.389822 24364 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 02:52:49.393301 24364 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 02:52:49.398473 24364 catalog_manager.cc:1347] Generated new cluster ID: 75075ea95f9b4226ba8bfd9968193af1
I20260811 02:52:49.398548 24364 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 02:52:49.414778 24364 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 02:52:49.415128 24364 catalog_manager.cc:1504] Loading token signing keys...
I20260811 02:52:49.421319 24364 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7c645129730f4b278d785393d40feb1f: Generated new TSK 0
I20260811 02:52:49.422019 24364 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 02:52:49.430833 12843 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDawEwS/build/debug/bin/kudu
/tmp/dist-test-taskDawEwS/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786416707705494-12843-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786416707705494-12843-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786416707705494-12843-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786416707705494-12843-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.138.193:0
--local_ip_for_outbound_sockets=127.12.138.193
--webserver_interface=127.12.138.193
--webserver_port=0
--tserver_master_addrs=127.12.138.254:42005
--builtin_ntp_servers=127.12.138.212:35887
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260811 02:52:49.549853 24376 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 02:52:49.550122 24376 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 02:52:49.550163 24376 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 02:52:49.554332 24376 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 02:52:49.554389 24376 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 02:52:49.554471 24376 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.138.193
I20260811 02:52:49.559342 24376 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.138.212:35887
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786416707705494-12843-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786416707705494-12843-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.138.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786416707705494-12843-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.12.138.193
--webserver_port=0
--tserver_master_addrs=127.12.138.254:42005
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24376
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.138.193
--log_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786416707705494-12843-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 02:09:49 UTC on ccycloud.quasar-fsnqqc.root.comops.site
build id 4579570
I20260811 02:52:49.560472 24376 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 02:52:49.561978 24376 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 02:52:49.571923 24381 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:52:49.572049 24384 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:52:49.571930 24382 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:52:49.572279 24376 server_base.cc:1034] running on GCE node
I20260811 02:52:49.572747 24376 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 02:52:49.573489 24376 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 02:52:49.574676 24376 hybrid_clock.cc:648] HybridClock initialized: now 1786416769574646 us; error 62 us; skew 500 ppm
I20260811 02:52:49.576650 24376 webserver.cc:460] Webserver started at http://127.12.138.193:42649/ using document root <none> and password file <none>
I20260811 02:52:49.577363 24376 fs_manager.cc:359] Metadata directory not provided
I20260811 02:52:49.577462 24376 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 02:52:49.577693 24376 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 02:52:49.579790 24376 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786416707705494-12843-0/minicluster-data/ts-0/data/instance:
uuid: "21ef6bb1a4f244c08c9379cea9b6d000"
format_stamp: "Formatted at 2026-08-11 02:52:49 on dist-test-slave-9gcw"
server_key: "02bebc6710c87ced1be9c95eb19d1805"
server_key_iv: "773f2a9b2486ca059f751db777043205"
server_key_version: "encryptionkey@0"
I20260811 02:52:49.580482 24376 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786416707705494-12843-0/minicluster-data/ts-0/wal/instance:
uuid: "21ef6bb1a4f244c08c9379cea9b6d000"
format_stamp: "Formatted at 2026-08-11 02:52:49 on dist-test-slave-9gcw"
server_key: "02bebc6710c87ced1be9c95eb19d1805"
server_key_iv: "773f2a9b2486ca059f751db777043205"
server_key_version: "encryptionkey@0"
I20260811 02:52:49.584421 24376 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260811 02:52:49.587275 24392 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 02:52:49.588759 24376 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260811 02:52:49.588909 24376 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786416707705494-12843-0/minicluster-data/ts-0/data,/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786416707705494-12843-0/minicluster-data/ts-0/wal
uuid: "21ef6bb1a4f244c08c9379cea9b6d000"
format_stamp: "Formatted at 2026-08-11 02:52:49 on dist-test-slave-9gcw"
server_key: "02bebc6710c87ced1be9c95eb19d1805"
server_key_iv: "773f2a9b2486ca059f751db777043205"
server_key_version: "encryptionkey@0"
I20260811 02:52:49.589036 24376 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786416707705494-12843-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786416707705494-12843-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786416707705494-12843-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 02:52:49.594072 24376 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 02:52:49.594658 24376 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 02:52:49.594802 24376 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 02:52:49.595314 24376 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 02:52:49.596269 24376 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 02:52:49.596334 24376 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 02:52:49.596372 24376 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 02:52:49.596388 24376 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 02:52:49.606307 24376 rpc_server.cc:307] RPC server started. Bound to: 127.12.138.193:33621
I20260811 02:52:49.606316 24521 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.138.193:33621 every 8 connection(s)
I20260811 02:52:49.607447 24376 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786416707705494-12843-0/minicluster-data/ts-0/data/info.pb
I20260811 02:52:49.608584 12843 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDawEwS/build/debug/bin/kudu as pid 24376
I20260811 02:52:49.608728 12843 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786416707705494-12843-0/minicluster-data/ts-0/wal/instance
I20260811 02:52:49.609124 12843 external_mini_cluster.cc:1428] Setting key 2894964d3ae256c731c3e3749bb7322f
I20260811 02:52:49.611531 12843 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDawEwS/build/debug/bin/kudu
/tmp/dist-test-taskDawEwS/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786416707705494-12843-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786416707705494-12843-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786416707705494-12843-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786416707705494-12843-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.138.194:0
--local_ip_for_outbound_sockets=127.12.138.194
--webserver_interface=127.12.138.194
--webserver_port=0
--tserver_master_addrs=127.12.138.254:42005
--builtin_ntp_servers=127.12.138.212:35887
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
I20260811 02:52:49.616148 24352 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:52:49.609781 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.138.193:38325 (local address 127.12.138.254:42005)
0811 02:52:49.609924 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0811 02:52:49.609928 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 02:52:49.611665 (+ 1737us) server_negotiation.cc:407] Connection header received
0811 02:52:49.614011 (+ 2346us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:52:49.614015 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:52:49.614070 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:52:49.614176 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:52:49.615620 (+ 1444us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:52:49.615623 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:52:49.615625 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 02:52:49.615639 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:52:49.615664 (+ 25us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 02:52:49.615673 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:52:49.615778 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0811 02:52:49.615984 (+ 206us) server_negotiation.cc:299] Negotiation successful
0811 02:52:49.616027 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260811 02:52:49.617612 24524 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:52:49.610223 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.138.254:42005 (local address 127.12.138.193:38325)
0811 02:52:49.611454 (+ 1231us) negotiation.cc:107] Waiting for socket to connect
0811 02:52:49.611516 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0811 02:52:49.613685 (+ 2169us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:52:49.614380 (+ 695us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:52:49.614388 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:52:49.614492 (+ 104us) client_negotiation.cc:189] Negotiated authn=SASL
0811 02:52:49.615230 (+ 738us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 02:52:49.615233 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0811 02:52:49.615298 (+ 65us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 02:52:49.615430 (+ 132us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 02:52:49.615802 (+ 372us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 02:52:49.615816 (+ 14us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 02:52:49.615826 (+ 10us) client_negotiation.cc:769] Sending connection context
0811 02:52:49.616034 (+ 208us) client_negotiation.cc:240] Negotiation successful
0811 02:52:49.616337 (+ 303us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":685,"thread_start_us":156,"threads_started":1}
I20260811 02:52:49.619109 24522 heartbeater.cc:347] Connected to a master server at 127.12.138.254:42005
I20260811 02:52:49.619450 24522 heartbeater.cc:464] Registering TS with master...
I20260811 02:52:49.620066 24522 heartbeater.cc:511] Master 127.12.138.254:42005 requested a full tablet report, sending...
I20260811 02:52:49.622237 24305 ts_manager.cc:194] Registered new tserver with Master: 21ef6bb1a4f244c08c9379cea9b6d000 (127.12.138.193:33621)
W20260811 02:52:49.736205 24525 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 02:52:49.736495 24525 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 02:52:49.736538 24525 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 02:52:49.740862 24525 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 02:52:49.740998 24525 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 02:52:49.741112 24525 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.138.194
I20260811 02:52:49.746115 24525 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.138.212:35887
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786416707705494-12843-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786416707705494-12843-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.138.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786416707705494-12843-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.12.138.194
--webserver_port=0
--tserver_master_addrs=127.12.138.254:42005
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24525
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.138.194
--log_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786416707705494-12843-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 02:09:49 UTC on ccycloud.quasar-fsnqqc.root.comops.site
build id 4579570
I20260811 02:52:49.747275 24525 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 02:52:49.748715 24525 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 02:52:49.758728 24531 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:52:49.758726 24532 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:52:49.758848 24534 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:52:49.759205 24525 server_base.cc:1034] running on GCE node
I20260811 02:52:49.759652 24525 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 02:52:49.760350 24525 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 02:52:49.761519 24525 hybrid_clock.cc:648] HybridClock initialized: now 1786416769761505 us; error 36 us; skew 500 ppm
I20260811 02:52:49.763653 24525 webserver.cc:460] Webserver started at http://127.12.138.194:38555/ using document root <none> and password file <none>
I20260811 02:52:49.764377 24525 fs_manager.cc:359] Metadata directory not provided
I20260811 02:52:49.764451 24525 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 02:52:49.764720 24525 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 02:52:49.767006 24525 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786416707705494-12843-0/minicluster-data/ts-1/data/instance:
uuid: "8d8d2bebd7e24ea5bddbebfd2a1dfe97"
format_stamp: "Formatted at 2026-08-11 02:52:49 on dist-test-slave-9gcw"
server_key: "9ade3b7f613c0d775d7753b09ab932df"
server_key_iv: "6f93071d6d9c64e48a4d99d0b4422e4e"
server_key_version: "encryptionkey@0"
I20260811 02:52:49.767674 24525 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786416707705494-12843-0/minicluster-data/ts-1/wal/instance:
uuid: "8d8d2bebd7e24ea5bddbebfd2a1dfe97"
format_stamp: "Formatted at 2026-08-11 02:52:49 on dist-test-slave-9gcw"
server_key: "9ade3b7f613c0d775d7753b09ab932df"
server_key_iv: "6f93071d6d9c64e48a4d99d0b4422e4e"
server_key_version: "encryptionkey@0"
I20260811 02:52:49.771832 24525 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.002s
I20260811 02:52:49.774709 24540 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 02:52:49.776206 24525 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260811 02:52:49.776386 24525 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786416707705494-12843-0/minicluster-data/ts-1/data,/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786416707705494-12843-0/minicluster-data/ts-1/wal
uuid: "8d8d2bebd7e24ea5bddbebfd2a1dfe97"
format_stamp: "Formatted at 2026-08-11 02:52:49 on dist-test-slave-9gcw"
server_key: "9ade3b7f613c0d775d7753b09ab932df"
server_key_iv: "6f93071d6d9c64e48a4d99d0b4422e4e"
server_key_version: "encryptionkey@0"
I20260811 02:52:49.776530 24525 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786416707705494-12843-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786416707705494-12843-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786416707705494-12843-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 02:52:49.781919 24525 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 02:52:49.782657 24525 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 02:52:49.782914 24525 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 02:52:49.783565 24525 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 02:52:49.784633 24525 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 02:52:49.784714 24525 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 02:52:49.784795 24525 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 02:52:49.784844 24525 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 02:52:49.795879 24525 rpc_server.cc:307] RPC server started. Bound to: 127.12.138.194:37501
I20260811 02:52:49.795898 24667 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.138.194:37501 every 8 connection(s)
I20260811 02:52:49.796967 24525 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786416707705494-12843-0/minicluster-data/ts-1/data/info.pb
I20260811 02:52:49.801209 12843 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDawEwS/build/debug/bin/kudu as pid 24525
I20260811 02:52:49.801369 12843 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786416707705494-12843-0/minicluster-data/ts-1/wal/instance
I20260811 02:52:49.801697 12843 external_mini_cluster.cc:1428] Setting key b0f411554b16275d775d799ab09318f5
I20260811 02:52:49.804049 12843 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDawEwS/build/debug/bin/kudu
/tmp/dist-test-taskDawEwS/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786416707705494-12843-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786416707705494-12843-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786416707705494-12843-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786416707705494-12843-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.138.195:0
--local_ip_for_outbound_sockets=127.12.138.195
--webserver_interface=127.12.138.195
--webserver_port=0
--tserver_master_addrs=127.12.138.254:42005
--builtin_ntp_servers=127.12.138.212:35887
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
I20260811 02:52:49.804222 24352 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:52:49.799237 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.138.194:38375 (local address 127.12.138.254:42005)
0811 02:52:49.799440 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0811 02:52:49.799444 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 02:52:49.800604 (+ 1160us) server_negotiation.cc:407] Connection header received
0811 02:52:49.802225 (+ 1621us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:52:49.802228 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:52:49.802302 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:52:49.802408 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:52:49.803699 (+ 1291us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:52:49.803703 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:52:49.803705 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 02:52:49.803720 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:52:49.803749 (+ 29us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 02:52:49.803762 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:52:49.803866 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0811 02:52:49.803992 (+ 126us) server_negotiation.cc:299] Negotiation successful
0811 02:52:49.804039 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260811 02:52:49.805058 24671 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:52:49.799690 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.138.254:42005 (local address 127.12.138.194:38375)
0811 02:52:49.800451 (+ 761us) negotiation.cc:107] Waiting for socket to connect
0811 02:52:49.800492 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0811 02:52:49.801960 (+ 1468us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:52:49.802631 (+ 671us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:52:49.802640 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:52:49.802753 (+ 113us) client_negotiation.cc:189] Negotiated authn=SASL
0811 02:52:49.803418 (+ 665us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 02:52:49.803420 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0811 02:52:49.803462 (+ 42us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 02:52:49.803532 (+ 70us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 02:52:49.803854 (+ 322us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 02:52:49.803861 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 02:52:49.803865 (+ 4us) client_negotiation.cc:769] Sending connection context
0811 02:52:49.803997 (+ 132us) client_negotiation.cc:240] Negotiation successful
0811 02:52:49.804246 (+ 249us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":340,"thread_start_us":167,"threads_started":1}
I20260811 02:52:49.806387 24668 heartbeater.cc:347] Connected to a master server at 127.12.138.254:42005
I20260811 02:52:49.806640 24668 heartbeater.cc:464] Registering TS with master...
I20260811 02:52:49.807083 24668 heartbeater.cc:511] Master 127.12.138.254:42005 requested a full tablet report, sending...
I20260811 02:52:49.808252 24305 ts_manager.cc:194] Registered new tserver with Master: 8d8d2bebd7e24ea5bddbebfd2a1dfe97 (127.12.138.194:37501)
W20260811 02:52:49.926450 24672 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 02:52:49.926736 24672 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 02:52:49.926779 24672 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 02:52:49.931012 24672 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 02:52:49.931071 24672 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 02:52:49.931172 24672 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.138.195
I20260811 02:52:49.936133 24672 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.138.212:35887
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786416707705494-12843-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786416707705494-12843-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.138.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786416707705494-12843-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.12.138.195
--webserver_port=0
--tserver_master_addrs=127.12.138.254:42005
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24672
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.138.195
--log_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786416707705494-12843-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 02:09:49 UTC on ccycloud.quasar-fsnqqc.root.comops.site
build id 4579570
I20260811 02:52:49.937343 24672 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 02:52:49.938827 24672 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 02:52:49.948783 24677 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:52:49.948904 24680 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:52:49.948779 24678 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:52:49.949376 24672 server_base.cc:1034] running on GCE node
I20260811 02:52:49.949803 24672 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 02:52:49.950481 24672 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 02:52:49.951680 24672 hybrid_clock.cc:648] HybridClock initialized: now 1786416769951645 us; error 46 us; skew 500 ppm
I20260811 02:52:49.953850 24672 webserver.cc:460] Webserver started at http://127.12.138.195:43059/ using document root <none> and password file <none>
I20260811 02:52:49.954507 24672 fs_manager.cc:359] Metadata directory not provided
I20260811 02:52:49.954650 24672 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 02:52:49.954916 24672 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 02:52:49.957316 24672 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786416707705494-12843-0/minicluster-data/ts-2/data/instance:
uuid: "04f4e18520dc4b338c62af181580ec5e"
format_stamp: "Formatted at 2026-08-11 02:52:49 on dist-test-slave-9gcw"
server_key: "21ea0853fe30db643553211f8bb87248"
server_key_iv: "9471e45e4f8eeb17b090df8bf05892fd"
server_key_version: "encryptionkey@0"
I20260811 02:52:49.957978 24672 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786416707705494-12843-0/minicluster-data/ts-2/wal/instance:
uuid: "04f4e18520dc4b338c62af181580ec5e"
format_stamp: "Formatted at 2026-08-11 02:52:49 on dist-test-slave-9gcw"
server_key: "21ea0853fe30db643553211f8bb87248"
server_key_iv: "9471e45e4f8eeb17b090df8bf05892fd"
server_key_version: "encryptionkey@0"
I20260811 02:52:49.962328 24672 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260811 02:52:49.965317 24687 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 02:52:49.966533 24672 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.001s
I20260811 02:52:49.966676 24672 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786416707705494-12843-0/minicluster-data/ts-2/data,/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786416707705494-12843-0/minicluster-data/ts-2/wal
uuid: "04f4e18520dc4b338c62af181580ec5e"
format_stamp: "Formatted at 2026-08-11 02:52:49 on dist-test-slave-9gcw"
server_key: "21ea0853fe30db643553211f8bb87248"
server_key_iv: "9471e45e4f8eeb17b090df8bf05892fd"
server_key_version: "encryptionkey@0"
I20260811 02:52:49.966807 24672 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786416707705494-12843-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786416707705494-12843-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786416707705494-12843-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 02:52:49.971705 24672 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 02:52:49.972375 24672 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 02:52:49.972558 24672 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 02:52:49.973100 24672 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 02:52:49.974086 24672 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 02:52:49.974159 24672 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 02:52:49.974226 24672 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 02:52:49.974282 24672 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 02:52:49.983561 24672 rpc_server.cc:307] RPC server started. Bound to: 127.12.138.195:44719
I20260811 02:52:49.983583 24825 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.138.195:44719 every 8 connection(s)
I20260811 02:52:49.984521 24672 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786416707705494-12843-0/minicluster-data/ts-2/data/info.pb
I20260811 02:52:49.991757 24352 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:52:49.986557 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.138.195:50183 (local address 127.12.138.254:42005)
0811 02:52:49.986762 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0811 02:52:49.986766 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 02:52:49.987928 (+ 1162us) server_negotiation.cc:407] Connection header received
0811 02:52:49.989654 (+ 1726us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:52:49.989658 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:52:49.989712 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:52:49.989828 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:52:49.991191 (+ 1363us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:52:49.991195 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:52:49.991198 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 02:52:49.991212 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:52:49.991236 (+ 24us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 02:52:49.991249 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:52:49.991365 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0811 02:52:49.991538 (+ 173us) server_negotiation.cc:299] Negotiation successful
0811 02:52:49.991589 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":105}
I20260811 02:52:49.992548 12843 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDawEwS/build/debug/bin/kudu as pid 24672
I20260811 02:52:49.992655 12843 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786416707705494-12843-0/minicluster-data/ts-2/wal/instance
I20260811 02:52:49.992789 24828 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:52:49.986952 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.138.254:42005 (local address 127.12.138.195:50183)
0811 02:52:49.987736 (+ 784us) negotiation.cc:107] Waiting for socket to connect
0811 02:52:49.987799 (+ 63us) client_negotiation.cc:174] Beginning negotiation
0811 02:52:49.989357 (+ 1558us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:52:49.990021 (+ 664us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:52:49.990028 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:52:49.990132 (+ 104us) client_negotiation.cc:189] Negotiated authn=SASL
0811 02:52:49.990883 (+ 751us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 02:52:49.990885 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0811 02:52:49.990941 (+ 56us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 02:52:49.991016 (+ 75us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 02:52:49.991356 (+ 340us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 02:52:49.991366 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 02:52:49.991374 (+ 8us) client_negotiation.cc:769] Sending connection context
0811 02:52:49.991586 (+ 212us) client_negotiation.cc:240] Negotiation successful
0811 02:52:49.991856 (+ 270us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":265,"thread_start_us":105,"threads_started":1}
I20260811 02:52:49.993109 12843 external_mini_cluster.cc:1428] Setting key 0bc02279d41af14e1f790b35a1925862
I20260811 02:52:49.994211 24826 heartbeater.cc:347] Connected to a master server at 127.12.138.254:42005
I20260811 02:52:49.994499 24826 heartbeater.cc:464] Registering TS with master...
I20260811 02:52:49.995065 24826 heartbeater.cc:511] Master 127.12.138.254:42005 requested a full tablet report, sending...
I20260811 02:52:49.996199 24305 ts_manager.cc:194] Registered new tserver with Master: 04f4e18520dc4b338c62af181580ec5e (127.12.138.195:44719)
I20260811 02:52:50.008008 12843 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260811 02:52:50.011288 24352 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 02:52:50.010196 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43494 (local address 127.12.138.254:42005)
0811 02:52:50.010374 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0811 02:52:50.010386 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 02:52:50.010534 (+ 148us) server_negotiation.cc:407] Connection header received
0811 02:52:50.010699 (+ 165us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:52:50.010702 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:52:50.010739 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:52:50.010816 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:52:50.011116 (+ 300us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43494: BlockingRecv error: recv got EOF from 127.0.0.1:43494 (error 108)
Metrics: {"server-negotiator.queue_time_us":76}
I20260811 02:52:50.011354 12843 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDawEwS/build/debug/bin/kudu with pid 24376
I20260811 02:52:50.020072 12843 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDawEwS/build/debug/bin/kudu with pid 24525
I20260811 02:52:50.029760 12843 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDawEwS/build/debug/bin/kudu with pid 24672
I20260811 02:52:50.038497 12843 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDawEwS/build/debug/bin/kudu with pid 24269
2026-08-11T02:52:50Z chronyd exiting
[ OK ] SecurityITest.TestRequireEncryptionInsecureCluster (883 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationSecureCluster
Loading random data
Initializing database '/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786416707705494-12843-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 02:52:50 dist-test-slave-9gcw krb5kdc[24836](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 02:52:50 dist-test-slave-9gcw krb5kdc[24836](info): set up 2 sockets
Aug 11 02:52:50 dist-test-slave-9gcw krb5kdc[24836](info): commencing operation
krb5kdc: starting...
W20260811 02:52:51.204694 12843 mini_kdc.cc:121] Time spent starting KDC: real 1.142s user 0.002s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786416707705494-12843-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786416707705494-12843-0/krb5kdc/test-user.keytab.
Aug 11 02:52:51 dist-test-slave-9gcw krb5kdc[24836](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416771, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T02:52:51Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T02:52:51Z Disabled control of system clock
WARNING: no policy specified for kudu/127.12.138.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.138.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.138.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786416707705494-12843-0/krb5kdc/kudu_127.12.138.254.keytab.
Entry for principal kudu/127.12.138.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786416707705494-12843-0/krb5kdc/kudu_127.12.138.254.keytab.
I20260811 02:52:51.353528 12843 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDawEwS/build/debug/bin/kudu
/tmp/dist-test-taskDawEwS/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786416707705494-12843-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786416707705494-12843-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786416707705494-12843-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786416707705494-12843-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.138.254:40431
--webserver_interface=127.12.138.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786416707705494-12843-0/krb5kdc/kudu_127.12.138.254.keytab
--principal=kudu/127.12.138.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.138.212:38967
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.12.138.254:40431
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786416707705494-12843-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786416707705494-12843-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786416707705494-12843-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 02:52:51.483145 24851 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 02:52:51.483520 24851 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 02:52:51.483635 24851 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 02:52:51.489190 24851 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 02:52:51.489296 24851 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 02:52:51.489337 24851 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 02:52:51.489396 24851 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 02:52:51.489437 24851 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 02:52:51.495714 24851 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.138.212:38967
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786416707705494-12843-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786416707705494-12843-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.12.138.254:40431
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786416707705494-12843-0/krb5kdc/kudu_127.12.138.254.keytab
--principal=kudu/127.12.138.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.138.254:40431
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786416707705494-12843-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.12.138.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24851
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786416707705494-12843-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 02:09:49 UTC on ccycloud.quasar-fsnqqc.root.comops.site
build id 4579570
I20260811 02:52:51.497015 24851 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 02:52:51.498193 24851 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 02:52:51.507737 24856 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:52:51.507783 24857 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:52:51.507870 24859 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:52:51.507982 24851 server_base.cc:1034] running on GCE node
I20260811 02:52:51.508641 24851 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 02:52:51.509992 24851 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 02:52:51.511215 24851 hybrid_clock.cc:648] HybridClock initialized: now 1786416771511195 us; error 29 us; skew 500 ppm
Aug 11 02:52:51 dist-test-slave-9gcw krb5kdc[24836](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416771, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.138.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 02:52:51.514472 24851 init.cc:377] Logged in from keytab as kudu/127.12.138.254@KRBTEST.COM (short username kudu)
I20260811 02:52:51.515517 24851 webserver.cc:460] Webserver started at http://127.12.138.254:37357/ using document root <none> and password file <none>
I20260811 02:52:51.516089 24851 fs_manager.cc:359] Metadata directory not provided
I20260811 02:52:51.516156 24851 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 02:52:51.516341 24851 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 02:52:51.518580 24851 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786416707705494-12843-0/minicluster-data/master-0/data/instance:
uuid: "c01c5e4e2fe0402e953959a5d5dccfa9"
format_stamp: "Formatted at 2026-08-11 02:52:51 on dist-test-slave-9gcw"
server_key: "8773d053f5e4f12d6aeb5deaa8213b39"
server_key_iv: "3c6c7ec09be8ba84773a1f2cb1198f3e"
server_key_version: "encryptionkey@0"
I20260811 02:52:51.519156 24851 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786416707705494-12843-0/minicluster-data/master-0/wal/instance:
uuid: "c01c5e4e2fe0402e953959a5d5dccfa9"
format_stamp: "Formatted at 2026-08-11 02:52:51 on dist-test-slave-9gcw"
server_key: "8773d053f5e4f12d6aeb5deaa8213b39"
server_key_iv: "3c6c7ec09be8ba84773a1f2cb1198f3e"
server_key_version: "encryptionkey@0"
I20260811 02:52:51.523216 24851 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260811 02:52:51.525959 24869 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 02:52:51.527372 24851 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 02:52:51.527489 24851 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786416707705494-12843-0/minicluster-data/master-0/data,/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786416707705494-12843-0/minicluster-data/master-0/wal
uuid: "c01c5e4e2fe0402e953959a5d5dccfa9"
format_stamp: "Formatted at 2026-08-11 02:52:51 on dist-test-slave-9gcw"
server_key: "8773d053f5e4f12d6aeb5deaa8213b39"
server_key_iv: "3c6c7ec09be8ba84773a1f2cb1198f3e"
server_key_version: "encryptionkey@0"
I20260811 02:52:51.527587 24851 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786416707705494-12843-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786416707705494-12843-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786416707705494-12843-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 02:52:51.557595 24851 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 02:52:51.560905 24851 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 02:52:51.561087 24851 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 02:52:51.569139 24851 rpc_server.cc:307] RPC server started. Bound to: 127.12.138.254:40431
I20260811 02:52:51.569479 24931 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.138.254:40431 every 8 connection(s)
I20260811 02:52:51.570293 24851 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786416707705494-12843-0/minicluster-data/master-0/data/info.pb
I20260811 02:52:51.572306 12843 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDawEwS/build/debug/bin/kudu as pid 24851
I20260811 02:52:51.572433 12843 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786416707705494-12843-0/minicluster-data/master-0/wal/instance
I20260811 02:52:51.572749 12843 external_mini_cluster.cc:1428] Setting key ad59fa79dfcedb0740c177c0820b1113
I20260811 02:52:51.573561 24932 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 02:52:51.579890 24932 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c01c5e4e2fe0402e953959a5d5dccfa9: Bootstrap starting.
I20260811 02:52:51.582450 24932 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c01c5e4e2fe0402e953959a5d5dccfa9: Neither blocks nor log segments found. Creating new log.
Aug 11 02:52:51 dist-test-slave-9gcw krb5kdc[24836](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416771, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.12.138.254@KRBTEST.COM
I20260811 02:52:51.583231 24932 log.cc:824] T 00000000000000000000000000000000 P c01c5e4e2fe0402e953959a5d5dccfa9: Log is configured to *not* fsync() on all Append() calls
I20260811 02:52:51.585106 24932 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c01c5e4e2fe0402e953959a5d5dccfa9: No bootstrap required, opened a new log
I20260811 02:52:51.587925 24935 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:52:51.574576 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40464 (local address 127.12.138.254:40431)
0811 02:52:51.574997 (+ 421us) server_negotiation.cc:206] Beginning negotiation
0811 02:52:51.575020 (+ 23us) server_negotiation.cc:399] Waiting for connection header
0811 02:52:51.575055 (+ 35us) server_negotiation.cc:407] Connection header received
0811 02:52:51.576044 (+ 989us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:52:51.576066 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:52:51.576409 (+ 343us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:52:51.576774 (+ 365us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:52:51.577683 (+ 909us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:52:51.578734 (+ 1051us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:52:51.579622 (+ 888us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:52:51.580170 (+ 548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:52:51.580346 (+ 176us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:52:51.583207 (+ 2861us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:52:51.583212 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:52:51.583225 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:52:51.583253 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:52:51.585029 (+ 1776us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:52:51.585588 (+ 559us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:52:51.585596 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:52:51.585606 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:52:51.585728 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:52:51.586031 (+ 303us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:52:51.586034 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:52:51.586036 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:52:51.586471 (+ 435us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:52:51.586688 (+ 217us) server_negotiation.cc:1035] Waiting for connection context
0811 02:52:51.586936 (+ 248us) server_negotiation.cc:299] Negotiation successful
0811 02:52:51.587224 (+ 288us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":282,"thread_start_us":141,"threads_started":1}
I20260811 02:52:51.588899 24932 raft_consensus.cc:348] T 00000000000000000000000000000000 P c01c5e4e2fe0402e953959a5d5dccfa9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c01c5e4e2fe0402e953959a5d5dccfa9" member_type: VOTER last_known_addr { host: "127.12.138.254" port: 40431 } }
I20260811 02:52:51.589145 24932 raft_consensus.cc:374] T 00000000000000000000000000000000 P c01c5e4e2fe0402e953959a5d5dccfa9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 02:52:51.589251 24932 raft_consensus.cc:729] T 00000000000000000000000000000000 P c01c5e4e2fe0402e953959a5d5dccfa9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c01c5e4e2fe0402e953959a5d5dccfa9, State: Initialized, Role: FOLLOWER
I20260811 02:52:51.589679 24932 consensus_queue.cc:260] T 00000000000000000000000000000000 P c01c5e4e2fe0402e953959a5d5dccfa9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c01c5e4e2fe0402e953959a5d5dccfa9" member_type: VOTER last_known_addr { host: "127.12.138.254" port: 40431 } }
I20260811 02:52:51.589838 24932 raft_consensus.cc:388] T 00000000000000000000000000000000 P c01c5e4e2fe0402e953959a5d5dccfa9 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 02:52:51.589924 24932 raft_consensus.cc:482] T 00000000000000000000000000000000 P c01c5e4e2fe0402e953959a5d5dccfa9 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 02:52:51.590046 24932 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c01c5e4e2fe0402e953959a5d5dccfa9 [term 0 FOLLOWER]: Advancing to term 1
I20260811 02:52:51.591120 24932 raft_consensus.cc:504] T 00000000000000000000000000000000 P c01c5e4e2fe0402e953959a5d5dccfa9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c01c5e4e2fe0402e953959a5d5dccfa9" member_type: VOTER last_known_addr { host: "127.12.138.254" port: 40431 } }
I20260811 02:52:51.591452 24932 leader_election.cc:302] T 00000000000000000000000000000000 P c01c5e4e2fe0402e953959a5d5dccfa9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c01c5e4e2fe0402e953959a5d5dccfa9; no voters:
I20260811 02:52:51.591787 24932 leader_election.cc:288] T 00000000000000000000000000000000 P c01c5e4e2fe0402e953959a5d5dccfa9 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 02:52:51.591878 24937 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c01c5e4e2fe0402e953959a5d5dccfa9 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 02:52:51.592051 24937 raft_consensus.cc:686] T 00000000000000000000000000000000 P c01c5e4e2fe0402e953959a5d5dccfa9 [term 1 LEADER]: Becoming Leader. State: Replica: c01c5e4e2fe0402e953959a5d5dccfa9, State: Running, Role: LEADER
I20260811 02:52:51.592346 24937 consensus_queue.cc:237] T 00000000000000000000000000000000 P c01c5e4e2fe0402e953959a5d5dccfa9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c01c5e4e2fe0402e953959a5d5dccfa9" member_type: VOTER last_known_addr { host: "127.12.138.254" port: 40431 } }
I20260811 02:52:51.593060 24932 sys_catalog.cc:564] T 00000000000000000000000000000000 P c01c5e4e2fe0402e953959a5d5dccfa9 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 02:52:51.593619 24939 sys_catalog.cc:455] T 00000000000000000000000000000000 P c01c5e4e2fe0402e953959a5d5dccfa9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c01c5e4e2fe0402e953959a5d5dccfa9. Latest consensus state: current_term: 1 leader_uuid: "c01c5e4e2fe0402e953959a5d5dccfa9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c01c5e4e2fe0402e953959a5d5dccfa9" member_type: VOTER last_known_addr { host: "127.12.138.254" port: 40431 } } }
I20260811 02:52:51.593743 24939 sys_catalog.cc:458] T 00000000000000000000000000000000 P c01c5e4e2fe0402e953959a5d5dccfa9 [sys.catalog]: This master's current role is: LEADER
I20260811 02:52:51.594013 24938 sys_catalog.cc:455] T 00000000000000000000000000000000 P c01c5e4e2fe0402e953959a5d5dccfa9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c01c5e4e2fe0402e953959a5d5dccfa9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c01c5e4e2fe0402e953959a5d5dccfa9" member_type: VOTER last_known_addr { host: "127.12.138.254" port: 40431 } } }
I20260811 02:52:51.594129 24938 sys_catalog.cc:458] T 00000000000000000000000000000000 P c01c5e4e2fe0402e953959a5d5dccfa9 [sys.catalog]: This master's current role is: LEADER
I20260811 02:52:51.594156 24942 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 02:52:51.597394 24942 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 02:52:51.603017 24942 catalog_manager.cc:1347] Generated new cluster ID: bd262df3bb7d4d55bc9b1b4179c3a463
I20260811 02:52:51.603091 24942 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 02:52:51.634630 24942 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 02:52:51.635960 24942 catalog_manager.cc:1504] Loading token signing keys...
I20260811 02:52:51.640720 24942 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c01c5e4e2fe0402e953959a5d5dccfa9: Generated new TSK 0
I20260811 02:52:51.641595 24942 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.12.138.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.138.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.138.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786416707705494-12843-0/krb5kdc/kudu_127.12.138.193.keytab.
Entry for principal kudu/127.12.138.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786416707705494-12843-0/krb5kdc/kudu_127.12.138.193.keytab.
I20260811 02:52:51.677969 12843 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDawEwS/build/debug/bin/kudu
/tmp/dist-test-taskDawEwS/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786416707705494-12843-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786416707705494-12843-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786416707705494-12843-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786416707705494-12843-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.138.193:0
--local_ip_for_outbound_sockets=127.12.138.193
--webserver_interface=127.12.138.193
--webserver_port=0
--tserver_master_addrs=127.12.138.254:40431
--keytab_file=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786416707705494-12843-0/krb5kdc/kudu_127.12.138.193.keytab
--principal=kudu/127.12.138.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.138.212:38967
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786416707705494-12843-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786416707705494-12843-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786416707705494-12843-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 02:52:51.802646 24960 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 02:52:51.802948 24960 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 02:52:51.802997 24960 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 02:52:51.807201 24960 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 02:52:51.807283 24960 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 02:52:51.807349 24960 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.138.193
I20260811 02:52:51.812675 24960 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.138.212:38967
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786416707705494-12843-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786416707705494-12843-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786416707705494-12843-0/krb5kdc/kudu_127.12.138.193.keytab
--principal=kudu/127.12.138.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.138.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786416707705494-12843-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.12.138.193
--webserver_port=0
--tserver_master_addrs=127.12.138.254:40431
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24960
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.138.193
--log_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786416707705494-12843-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 02:09:49 UTC on ccycloud.quasar-fsnqqc.root.comops.site
build id 4579570
I20260811 02:52:51.813889 24960 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 02:52:51.814972 24960 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 02:52:51.825886 24966 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:52:51.825950 24965 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:52:51.825927 24968 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:52:51.826047 24960 server_base.cc:1034] running on GCE node
I20260811 02:52:51.826663 24960 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 02:52:51.827378 24960 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 02:52:51.828564 24960 hybrid_clock.cc:648] HybridClock initialized: now 1786416771828530 us; error 44 us; skew 500 ppm
Aug 11 02:52:51 dist-test-slave-9gcw krb5kdc[24836](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416771, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.138.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 02:52:51.832047 24960 init.cc:377] Logged in from keytab as kudu/127.12.138.193@KRBTEST.COM (short username kudu)
I20260811 02:52:51.833338 24960 webserver.cc:460] Webserver started at http://127.12.138.193:40053/ using document root <none> and password file <none>
I20260811 02:52:51.833984 24960 fs_manager.cc:359] Metadata directory not provided
I20260811 02:52:51.834092 24960 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 02:52:51.834357 24960 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 02:52:51.836738 24960 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786416707705494-12843-0/minicluster-data/ts-0/data/instance:
uuid: "3390c91c03de4b419bc50509131eacfa"
format_stamp: "Formatted at 2026-08-11 02:52:51 on dist-test-slave-9gcw"
server_key: "709c02c14db63aadff8f2994fa8893d7"
server_key_iv: "3a1ea377d50ea00857cd26a3a0c91ec6"
server_key_version: "encryptionkey@0"
I20260811 02:52:51.837455 24960 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786416707705494-12843-0/minicluster-data/ts-0/wal/instance:
uuid: "3390c91c03de4b419bc50509131eacfa"
format_stamp: "Formatted at 2026-08-11 02:52:51 on dist-test-slave-9gcw"
server_key: "709c02c14db63aadff8f2994fa8893d7"
server_key_iv: "3a1ea377d50ea00857cd26a3a0c91ec6"
server_key_version: "encryptionkey@0"
I20260811 02:52:51.841890 24960 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.000s sys 0.003s
I20260811 02:52:51.844748 24976 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 02:52:51.846293 24960 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260811 02:52:51.846472 24960 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786416707705494-12843-0/minicluster-data/ts-0/data,/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786416707705494-12843-0/minicluster-data/ts-0/wal
uuid: "3390c91c03de4b419bc50509131eacfa"
format_stamp: "Formatted at 2026-08-11 02:52:51 on dist-test-slave-9gcw"
server_key: "709c02c14db63aadff8f2994fa8893d7"
server_key_iv: "3a1ea377d50ea00857cd26a3a0c91ec6"
server_key_version: "encryptionkey@0"
I20260811 02:52:51.846657 24960 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786416707705494-12843-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786416707705494-12843-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786416707705494-12843-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 02:52:51.859452 24960 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 02:52:51.863337 24960 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 02:52:51.863560 24960 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 02:52:51.864233 24960 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 02:52:51.865118 24960 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 02:52:51.865231 24960 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 02:52:51.865315 24960 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 02:52:51.865347 24960 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 02:52:51.874712 24960 rpc_server.cc:307] RPC server started. Bound to: 127.12.138.193:38729
I20260811 02:52:51.874787 25106 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.138.193:38729 every 8 connection(s)
I20260811 02:52:51.875753 24960 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786416707705494-12843-0/minicluster-data/ts-0/data/info.pb
I20260811 02:52:51.876660 12843 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDawEwS/build/debug/bin/kudu as pid 24960
I20260811 02:52:51.876787 12843 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786416707705494-12843-0/minicluster-data/ts-0/wal/instance
I20260811 02:52:51.877133 12843 external_mini_cluster.cc:1428] Setting key 5ab628eb679c1087d5a503bed0a2b9fd
Aug 11 02:52:51 dist-test-slave-9gcw krb5kdc[24836](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416771, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.138.193@KRBTEST.COM for kudu/127.12.138.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.12.138.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.138.194@KRBTEST.COM" created.
I20260811 02:52:51.893564 24935 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:52:51.878017 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.138.193:58153 (local address 127.12.138.254:40431)
0811 02:52:51.878199 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0811 02:52:51.878204 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 02:52:51.879301 (+ 1097us) server_negotiation.cc:407] Connection header received
0811 02:52:51.881457 (+ 2156us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:52:51.881462 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:52:51.881529 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:52:51.881620 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:52:51.882892 (+ 1272us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:52:51.883488 (+ 596us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:52:51.885517 (+ 2029us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:52:51.885821 (+ 304us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:52:51.885928 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:52:51.889249 (+ 3321us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:52:51.889252 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:52:51.889254 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:52:51.889282 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:52:51.890763 (+ 1481us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:52:51.891525 (+ 762us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:52:51.891529 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:52:51.891530 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:52:51.891654 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:52:51.892114 (+ 460us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:52:51.892118 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:52:51.892119 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:52:51.892364 (+ 245us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:52:51.892500 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0811 02:52:51.893308 (+ 808us) server_negotiation.cc:299] Negotiation successful
0811 02:52:51.893432 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":94}
I20260811 02:52:51.895083 25109 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:52:51.878485 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.138.254:40431 (local address 127.12.138.193:58153)
0811 02:52:51.879118 (+ 633us) negotiation.cc:107] Waiting for socket to connect
0811 02:52:51.879177 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0811 02:52:51.881125 (+ 1948us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:52:51.881780 (+ 655us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:52:51.881786 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:52:51.882151 (+ 365us) client_negotiation.cc:189] Negotiated authn=SASL
0811 02:52:51.882693 (+ 542us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:52:51.882718 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:52:51.883628 (+ 910us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:52:51.883634 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:52:51.885350 (+ 1716us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:52:51.885367 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:52:51.886008 (+ 641us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:52:51.886016 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:52:51.886274 (+ 258us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:52:51.887017 (+ 743us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 02:52:51.887053 (+ 36us) client_negotiation.cc:656] calling sasl_client_start()
0811 02:52:51.888990 (+ 1937us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 02:52:51.890901 (+ 1911us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:52:51.890907 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:52:51.890924 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:52:51.891356 (+ 432us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:52:51.891781 (+ 425us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:52:51.891784 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:52:51.891786 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:52:51.891951 (+ 165us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:52:51.892498 (+ 547us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 02:52:51.892506 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 02:52:51.892966 (+ 460us) client_negotiation.cc:769] Sending connection context
0811 02:52:51.893266 (+ 300us) client_negotiation.cc:240] Negotiation successful
0811 02:52:51.893721 (+ 455us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":396,"thread_start_us":176,"threads_started":1}
I20260811 02:52:51.896744 25107 heartbeater.cc:347] Connected to a master server at 127.12.138.254:40431
I20260811 02:52:51.897037 25107 heartbeater.cc:464] Registering TS with master...
I20260811 02:52:51.897689 25107 heartbeater.cc:511] Master 127.12.138.254:40431 requested a full tablet report, sending...
I20260811 02:52:51.899747 24889 ts_manager.cc:194] Registered new tserver with Master: 3390c91c03de4b419bc50509131eacfa (127.12.138.193:38729)
I20260811 02:52:51.901510 24889 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.12.138.193@KRBTEST.COM'} at 127.12.138.193:58153
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.138.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786416707705494-12843-0/krb5kdc/kudu_127.12.138.194.keytab.
Entry for principal kudu/127.12.138.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786416707705494-12843-0/krb5kdc/kudu_127.12.138.194.keytab.
I20260811 02:52:51.913435 12843 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDawEwS/build/debug/bin/kudu
/tmp/dist-test-taskDawEwS/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786416707705494-12843-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786416707705494-12843-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786416707705494-12843-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786416707705494-12843-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.138.194:0
--local_ip_for_outbound_sockets=127.12.138.194
--webserver_interface=127.12.138.194
--webserver_port=0
--tserver_master_addrs=127.12.138.254:40431
--keytab_file=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786416707705494-12843-0/krb5kdc/kudu_127.12.138.194.keytab
--principal=kudu/127.12.138.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.138.212:38967
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786416707705494-12843-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786416707705494-12843-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786416707705494-12843-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 02:52:52.030488 25113 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 02:52:52.030797 25113 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 02:52:52.030875 25113 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 02:52:52.035116 25113 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 02:52:52.035182 25113 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 02:52:52.035277 25113 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.138.194
I20260811 02:52:52.040887 25113 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.138.212:38967
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786416707705494-12843-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786416707705494-12843-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786416707705494-12843-0/krb5kdc/kudu_127.12.138.194.keytab
--principal=kudu/127.12.138.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.138.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786416707705494-12843-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.12.138.194
--webserver_port=0
--tserver_master_addrs=127.12.138.254:40431
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25113
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.138.194
--log_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786416707705494-12843-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 02:09:49 UTC on ccycloud.quasar-fsnqqc.root.comops.site
build id 4579570
I20260811 02:52:52.042151 25113 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 02:52:52.043558 25113 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260811 02:52:52.054181 25113 server_base.cc:1034] running on GCE node
W20260811 02:52:52.054102 25120 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:52:52.054080 25119 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:52:52.054131 25123 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:52:52.054883 25113 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 02:52:52.055593 25113 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 02:52:52.056813 25113 hybrid_clock.cc:648] HybridClock initialized: now 1786416772056795 us; error 32 us; skew 500 ppm
Aug 11 02:52:52 dist-test-slave-9gcw krb5kdc[24836](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416772, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.138.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 02:52:52.060226 25113 init.cc:377] Logged in from keytab as kudu/127.12.138.194@KRBTEST.COM (short username kudu)
I20260811 02:52:52.061281 25113 webserver.cc:460] Webserver started at http://127.12.138.194:34103/ using document root <none> and password file <none>
I20260811 02:52:52.061854 25113 fs_manager.cc:359] Metadata directory not provided
I20260811 02:52:52.061915 25113 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 02:52:52.062125 25113 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 02:52:52.064535 25113 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786416707705494-12843-0/minicluster-data/ts-1/data/instance:
uuid: "329fba0a0c4f4c639f4a8bb2449ccbd3"
format_stamp: "Formatted at 2026-08-11 02:52:52 on dist-test-slave-9gcw"
server_key: "149ae8e375dee183a5938ce2d4b08f32"
server_key_iv: "322cc9a5afe7e2a81d2633897b6c8385"
server_key_version: "encryptionkey@0"
I20260811 02:52:52.065099 25113 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786416707705494-12843-0/minicluster-data/ts-1/wal/instance:
uuid: "329fba0a0c4f4c639f4a8bb2449ccbd3"
format_stamp: "Formatted at 2026-08-11 02:52:52 on dist-test-slave-9gcw"
server_key: "149ae8e375dee183a5938ce2d4b08f32"
server_key_iv: "322cc9a5afe7e2a81d2633897b6c8385"
server_key_version: "encryptionkey@0"
I20260811 02:52:52.069399 25113 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260811 02:52:52.072058 25130 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 02:52:52.073601 25113 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 02:52:52.073724 25113 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786416707705494-12843-0/minicluster-data/ts-1/data,/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786416707705494-12843-0/minicluster-data/ts-1/wal
uuid: "329fba0a0c4f4c639f4a8bb2449ccbd3"
format_stamp: "Formatted at 2026-08-11 02:52:52 on dist-test-slave-9gcw"
server_key: "149ae8e375dee183a5938ce2d4b08f32"
server_key_iv: "322cc9a5afe7e2a81d2633897b6c8385"
server_key_version: "encryptionkey@0"
I20260811 02:52:52.073874 25113 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786416707705494-12843-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786416707705494-12843-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786416707705494-12843-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 02:52:52.085631 25113 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 02:52:52.089006 25113 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 02:52:52.089262 25113 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 02:52:52.089887 25113 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 02:52:52.090756 25113 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 02:52:52.090806 25113 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 02:52:52.090847 25113 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 02:52:52.090898 25113 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 02:52:52.101775 25113 rpc_server.cc:307] RPC server started. Bound to: 127.12.138.194:45375
I20260811 02:52:52.101812 25261 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.138.194:45375 every 8 connection(s)
I20260811 02:52:52.102756 25113 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786416707705494-12843-0/minicluster-data/ts-1/data/info.pb
I20260811 02:52:52.111260 12843 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDawEwS/build/debug/bin/kudu as pid 25113
I20260811 02:52:52.111426 12843 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786416707705494-12843-0/minicluster-data/ts-1/wal/instance
I20260811 02:52:52.111729 12843 external_mini_cluster.cc:1428] Setting key 3eb0c2c95ff4cba98fb9a6c8fe9aa518
Aug 11 02:52:52 dist-test-slave-9gcw krb5kdc[24836](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416772, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.138.194@KRBTEST.COM for kudu/127.12.138.254@KRBTEST.COM
I20260811 02:52:52.118912 24935 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:52:52.104733 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.138.194:59283 (local address 127.12.138.254:40431)
0811 02:52:52.104882 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0811 02:52:52.104886 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 02:52:52.105964 (+ 1078us) server_negotiation.cc:407] Connection header received
0811 02:52:52.107458 (+ 1494us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:52:52.107462 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:52:52.107525 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:52:52.107634 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:52:52.108881 (+ 1247us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:52:52.109485 (+ 604us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:52:52.110716 (+ 1231us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:52:52.110985 (+ 269us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:52:52.111114 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:52:52.114253 (+ 3139us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:52:52.114260 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:52:52.114266 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:52:52.114298 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:52:52.116147 (+ 1849us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:52:52.116895 (+ 748us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:52:52.116898 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:52:52.116899 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:52:52.116981 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:52:52.117473 (+ 492us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:52:52.117479 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:52:52.117484 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:52:52.117757 (+ 273us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:52:52.117897 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0811 02:52:52.118484 (+ 587us) server_negotiation.cc:299] Negotiation successful
0811 02:52:52.118644 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260811 02:52:52.119729 25265 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:52:52.105290 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.138.254:40431 (local address 127.12.138.194:59283)
0811 02:52:52.105788 (+ 498us) negotiation.cc:107] Waiting for socket to connect
0811 02:52:52.105840 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0811 02:52:52.107194 (+ 1354us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:52:52.107818 (+ 624us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:52:52.107825 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:52:52.108209 (+ 384us) client_negotiation.cc:189] Negotiated authn=SASL
0811 02:52:52.108719 (+ 510us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:52:52.108733 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:52:52.109625 (+ 892us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:52:52.109633 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:52:52.110571 (+ 938us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:52:52.110587 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:52:52.111159 (+ 572us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:52:52.111163 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:52:52.111374 (+ 211us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:52:52.112122 (+ 748us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 02:52:52.112139 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 02:52:52.114059 (+ 1920us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 02:52:52.116336 (+ 2277us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:52:52.116343 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:52:52.116365 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:52:52.116786 (+ 421us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:52:52.117082 (+ 296us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:52:52.117087 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:52:52.117091 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:52:52.117358 (+ 267us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:52:52.117898 (+ 540us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 02:52:52.117903 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 02:52:52.118178 (+ 275us) client_negotiation.cc:769] Sending connection context
0811 02:52:52.118375 (+ 197us) client_negotiation.cc:240] Negotiation successful
0811 02:52:52.118675 (+ 300us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":294,"thread_start_us":129,"threads_started":1}
I20260811 02:52:52.121109 25263 heartbeater.cc:347] Connected to a master server at 127.12.138.254:40431
I20260811 02:52:52.121441 25263 heartbeater.cc:464] Registering TS with master...
I20260811 02:52:52.122125 25263 heartbeater.cc:511] Master 127.12.138.254:40431 requested a full tablet report, sending...
I20260811 02:52:52.123453 24889 ts_manager.cc:194] Registered new tserver with Master: 329fba0a0c4f4c639f4a8bb2449ccbd3 (127.12.138.194:45375)
WARNING: no policy specified for kudu/127.12.138.195@KRBTEST.COM; defaulting to no policy
I20260811 02:52:52.124089 24889 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.12.138.194@KRBTEST.COM'} at 127.12.138.194:59283
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.138.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.138.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786416707705494-12843-0/krb5kdc/kudu_127.12.138.195.keytab.
Entry for principal kudu/127.12.138.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786416707705494-12843-0/krb5kdc/kudu_127.12.138.195.keytab.
I20260811 02:52:52.149134 12843 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDawEwS/build/debug/bin/kudu
/tmp/dist-test-taskDawEwS/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786416707705494-12843-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786416707705494-12843-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786416707705494-12843-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786416707705494-12843-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.138.195:0
--local_ip_for_outbound_sockets=127.12.138.195
--webserver_interface=127.12.138.195
--webserver_port=0
--tserver_master_addrs=127.12.138.254:40431
--keytab_file=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786416707705494-12843-0/krb5kdc/kudu_127.12.138.195.keytab
--principal=kudu/127.12.138.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.138.212:38967
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786416707705494-12843-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786416707705494-12843-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786416707705494-12843-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 02:52:52.268525 25269 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 02:52:52.268918 25269 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 02:52:52.269003 25269 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 02:52:52.273386 25269 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 02:52:52.273490 25269 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 02:52:52.273588 25269 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.138.195
I20260811 02:52:52.279917 25269 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.138.212:38967
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786416707705494-12843-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786416707705494-12843-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786416707705494-12843-0/krb5kdc/kudu_127.12.138.195.keytab
--principal=kudu/127.12.138.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.138.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786416707705494-12843-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.12.138.195
--webserver_port=0
--tserver_master_addrs=127.12.138.254:40431
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25269
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.138.195
--log_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786416707705494-12843-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 02:09:49 UTC on ccycloud.quasar-fsnqqc.root.comops.site
build id 4579570
I20260811 02:52:52.281243 25269 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 02:52:52.282373 25269 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 02:52:52.294304 25275 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:52:52.294386 25276 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:52:52.294304 25278 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:52:52.294981 25269 server_base.cc:1034] running on GCE node
I20260811 02:52:52.295528 25269 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 02:52:52.296222 25269 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 02:52:52.297406 25269 hybrid_clock.cc:648] HybridClock initialized: now 1786416772297329 us; error 90 us; skew 500 ppm
Aug 11 02:52:52 dist-test-slave-9gcw krb5kdc[24836](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416772, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.138.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 02:52:52.300719 25269 init.cc:377] Logged in from keytab as kudu/127.12.138.195@KRBTEST.COM (short username kudu)
I20260811 02:52:52.301779 25269 webserver.cc:460] Webserver started at http://127.12.138.195:44669/ using document root <none> and password file <none>
I20260811 02:52:52.302403 25269 fs_manager.cc:359] Metadata directory not provided
I20260811 02:52:52.302501 25269 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 02:52:52.302757 25269 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 02:52:52.305089 25269 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786416707705494-12843-0/minicluster-data/ts-2/data/instance:
uuid: "3cf081e80f20400bbf9fd1eaa41f9b95"
format_stamp: "Formatted at 2026-08-11 02:52:52 on dist-test-slave-9gcw"
server_key: "1759603cc44e8e8eee396fe85b772c5e"
server_key_iv: "80da718433a87428926d7d05ecd2e4b6"
server_key_version: "encryptionkey@0"
I20260811 02:52:52.305768 25269 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786416707705494-12843-0/minicluster-data/ts-2/wal/instance:
uuid: "3cf081e80f20400bbf9fd1eaa41f9b95"
format_stamp: "Formatted at 2026-08-11 02:52:52 on dist-test-slave-9gcw"
server_key: "1759603cc44e8e8eee396fe85b772c5e"
server_key_iv: "80da718433a87428926d7d05ecd2e4b6"
server_key_version: "encryptionkey@0"
I20260811 02:52:52.310034 25269 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260811 02:52:52.312647 25285 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 02:52:52.314044 25269 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 02:52:52.314188 25269 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786416707705494-12843-0/minicluster-data/ts-2/data,/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786416707705494-12843-0/minicluster-data/ts-2/wal
uuid: "3cf081e80f20400bbf9fd1eaa41f9b95"
format_stamp: "Formatted at 2026-08-11 02:52:52 on dist-test-slave-9gcw"
server_key: "1759603cc44e8e8eee396fe85b772c5e"
server_key_iv: "80da718433a87428926d7d05ecd2e4b6"
server_key_version: "encryptionkey@0"
I20260811 02:52:52.314311 25269 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786416707705494-12843-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786416707705494-12843-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786416707705494-12843-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 02:52:52.327088 25269 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 02:52:52.330524 25269 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 02:52:52.330780 25269 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 02:52:52.331405 25269 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 02:52:52.332321 25269 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 02:52:52.332396 25269 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 02:52:52.332471 25269 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 02:52:52.332526 25269 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 02:52:52.342304 25269 rpc_server.cc:307] RPC server started. Bound to: 127.12.138.195:34879
I20260811 02:52:52.342336 25419 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.138.195:34879 every 8 connection(s)
I20260811 02:52:52.343277 25269 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786416707705494-12843-0/minicluster-data/ts-2/data/info.pb
I20260811 02:52:52.347599 12843 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDawEwS/build/debug/bin/kudu as pid 25269
I20260811 02:52:52.347771 12843 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786416707705494-12843-0/minicluster-data/ts-2/wal/instance
I20260811 02:52:52.348089 12843 external_mini_cluster.cc:1428] Setting key 3d734a16ee64a4a4c41345c2715d0674
Aug 11 02:52:52 dist-test-slave-9gcw krb5kdc[24836](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416772, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.138.195@KRBTEST.COM for kudu/127.12.138.254@KRBTEST.COM
I20260811 02:52:52.357893 24935 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:52:52.345304 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.138.195:55581 (local address 127.12.138.254:40431)
0811 02:52:52.345451 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0811 02:52:52.345454 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 02:52:52.346345 (+ 891us) server_negotiation.cc:407] Connection header received
0811 02:52:52.347848 (+ 1503us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:52:52.347851 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:52:52.347918 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:52:52.348027 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:52:52.349516 (+ 1489us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:52:52.350176 (+ 660us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:52:52.351550 (+ 1374us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:52:52.351778 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:52:52.351896 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:52:52.354304 (+ 2408us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:52:52.354314 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:52:52.354317 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:52:52.354342 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:52:52.355721 (+ 1379us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:52:52.356260 (+ 539us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:52:52.356264 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:52:52.356266 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:52:52.356354 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:52:52.356714 (+ 360us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:52:52.356718 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:52:52.356720 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:52:52.356910 (+ 190us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:52:52.357021 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0811 02:52:52.357583 (+ 562us) server_negotiation.cc:299] Negotiation successful
0811 02:52:52.357713 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260811 02:52:52.358639 25424 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:52:52.345696 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.138.254:40431 (local address 127.12.138.195:55581)
0811 02:52:52.346180 (+ 484us) negotiation.cc:107] Waiting for socket to connect
0811 02:52:52.346222 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0811 02:52:52.347563 (+ 1341us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:52:52.348242 (+ 679us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:52:52.348257 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:52:52.348687 (+ 430us) client_negotiation.cc:189] Negotiated authn=SASL
0811 02:52:52.349374 (+ 687us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:52:52.349390 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:52:52.350308 (+ 918us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:52:52.350311 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:52:52.351417 (+ 1106us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:52:52.351427 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:52:52.351934 (+ 507us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:52:52.351939 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:52:52.352109 (+ 170us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:52:52.352650 (+ 541us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 02:52:52.352666 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0811 02:52:52.354132 (+ 1466us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 02:52:52.355864 (+ 1732us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:52:52.355870 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:52:52.355885 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:52:52.356145 (+ 260us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:52:52.356450 (+ 305us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:52:52.356455 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:52:52.356457 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:52:52.356608 (+ 151us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:52:52.357018 (+ 410us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 02:52:52.357024 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 02:52:52.357336 (+ 312us) client_negotiation.cc:769] Sending connection context
0811 02:52:52.357523 (+ 187us) client_negotiation.cc:240] Negotiation successful
0811 02:52:52.357841 (+ 318us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":287,"thread_start_us":130,"threads_started":1}
I20260811 02:52:52.360023 25420 heartbeater.cc:347] Connected to a master server at 127.12.138.254:40431
I20260811 02:52:52.360221 25420 heartbeater.cc:464] Registering TS with master...
I20260811 02:52:52.360661 25420 heartbeater.cc:511] Master 127.12.138.254:40431 requested a full tablet report, sending...
I20260811 02:52:52.361871 24889 ts_manager.cc:194] Registered new tserver with Master: 3cf081e80f20400bbf9fd1eaa41f9b95 (127.12.138.195:34879)
I20260811 02:52:52.362532 24889 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.12.138.195@KRBTEST.COM'} at 127.12.138.195:55581
I20260811 02:52:52.362723 12843 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 02:52:52.372426 24935 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:52:52.365076 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55454 (local address 127.12.138.254:40431)
0811 02:52:52.365269 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0811 02:52:52.365272 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 02:52:52.365398 (+ 126us) server_negotiation.cc:407] Connection header received
0811 02:52:52.365522 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:52:52.365524 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:52:52.365563 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:52:52.365627 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:52:52.366339 (+ 712us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:52:52.366820 (+ 481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:52:52.367671 (+ 851us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:52:52.367883 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:52:52.367962 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:52:52.369485 (+ 1523us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:52:52.369488 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:52:52.369490 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:52:52.369512 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:52:52.370851 (+ 1339us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:52:52.371248 (+ 397us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:52:52.371250 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:52:52.371252 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:52:52.371339 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:52:52.371591 (+ 252us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:52:52.371593 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:52:52.371594 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:52:52.371809 (+ 215us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:52:52.371913 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0811 02:52:52.372124 (+ 211us) server_negotiation.cc:299] Negotiation successful
0811 02:52:52.372222 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":110}
I20260811 02:52:52.375998 24889 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55454:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 02:52:52.379549 24889 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 02:52:52.395424 25437 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:52:52.388996 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.138.195:34879 (local address 127.0.0.1:54356)
0811 02:52:52.389602 (+ 606us) negotiation.cc:107] Waiting for socket to connect
0811 02:52:52.389651 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0811 02:52:52.389787 (+ 136us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:52:52.390628 (+ 841us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:52:52.390634 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:52:52.390658 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 02:52:52.390815 (+ 157us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:52:52.390823 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:52:52.392142 (+ 1319us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:52:52.392148 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:52:52.393616 (+ 1468us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:52:52.393628 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:52:52.394744 (+ 1116us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:52:52.394748 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:52:52.394856 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:52:52.394895 (+ 39us) client_negotiation.cc:769] Sending connection context
0811 02:52:52.395056 (+ 161us) client_negotiation.cc:240] Negotiation successful
0811 02:52:52.395204 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":524,"thread_start_us":135,"threads_started":1}
I20260811 02:52:52.395479 25434 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:52:52.388773 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.138.193:38729 (local address 127.0.0.1:40920)
0811 02:52:52.389575 (+ 802us) negotiation.cc:107] Waiting for socket to connect
0811 02:52:52.389629 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0811 02:52:52.389789 (+ 160us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:52:52.391016 (+ 1227us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:52:52.391020 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:52:52.391040 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 02:52:52.391189 (+ 149us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:52:52.391200 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:52:52.392754 (+ 1554us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:52:52.392758 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:52:52.393953 (+ 1195us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:52:52.393963 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:52:52.395073 (+ 1110us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:52:52.395076 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:52:52.395196 (+ 120us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:52:52.395227 (+ 31us) client_negotiation.cc:769] Sending connection context
0811 02:52:52.395297 (+ 70us) client_negotiation.cc:240] Negotiation successful
0811 02:52:52.395368 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":613,"mutex_wait_us":28,"thread_start_us":124,"threads_started":1}
I20260811 02:52:52.395643 25435 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:52:52.389028 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40920 (local address 127.12.138.193:38729)
0811 02:52:52.390537 (+ 1509us) server_negotiation.cc:206] Beginning negotiation
0811 02:52:52.390554 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0811 02:52:52.390578 (+ 24us) server_negotiation.cc:407] Connection header received
0811 02:52:52.390651 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:52:52.390667 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:52:52.390857 (+ 190us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:52:52.391000 (+ 143us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 02:52:52.391345 (+ 345us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:52:52.392581 (+ 1236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:52:52.394070 (+ 1489us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:52:52.394932 (+ 862us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:52:52.395098 (+ 166us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:52:52.395238 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0811 02:52:52.395368 (+ 130us) server_negotiation.cc:299] Negotiation successful
0811 02:52:52.395478 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1393,"thread_start_us":129,"threads_started":1}
I20260811 02:52:52.396178 25436 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:52:52.389352 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.138.194:45375 (local address 127.0.0.1:54980)
0811 02:52:52.390292 (+ 940us) negotiation.cc:107] Waiting for socket to connect
0811 02:52:52.390314 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0811 02:52:52.390436 (+ 122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:52:52.391171 (+ 735us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:52:52.391176 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:52:52.391199 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 02:52:52.391296 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:52:52.391302 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:52:52.392771 (+ 1469us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:52:52.392777 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:52:52.394259 (+ 1482us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:52:52.394272 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:52:52.395836 (+ 1564us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:52:52.395840 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:52:52.395923 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:52:52.395944 (+ 21us) client_negotiation.cc:769] Sending connection context
0811 02:52:52.396014 (+ 70us) client_negotiation.cc:240] Negotiation successful
0811 02:52:52.396069 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":878,"mutex_wait_us":22,"thread_start_us":102,"threads_started":1}
I20260811 02:52:52.396348 25439 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:52:52.389803 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54980 (local address 127.12.138.194:45375)
0811 02:52:52.390201 (+ 398us) server_negotiation.cc:206] Beginning negotiation
0811 02:52:52.390215 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0811 02:52:52.390545 (+ 330us) server_negotiation.cc:407] Connection header received
0811 02:52:52.390639 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:52:52.390648 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:52:52.390854 (+ 206us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:52:52.391048 (+ 194us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 02:52:52.391406 (+ 358us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:52:52.392603 (+ 1197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:52:52.394420 (+ 1817us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:52:52.395275 (+ 855us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:52:52.395429 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:52:52.395762 (+ 333us) server_negotiation.cc:1035] Waiting for connection context
0811 02:52:52.396087 (+ 325us) server_negotiation.cc:299] Negotiation successful
0811 02:52:52.396187 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":288,"thread_start_us":199,"threads_started":1}
I20260811 02:52:52.397164 25438 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:52:52.389671 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54356 (local address 127.12.138.195:34879)
0811 02:52:52.390086 (+ 415us) server_negotiation.cc:206] Beginning negotiation
0811 02:52:52.390100 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0811 02:52:52.390125 (+ 25us) server_negotiation.cc:407] Connection header received
0811 02:52:52.390208 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:52:52.390213 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:52:52.390437 (+ 224us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:52:52.390991 (+ 554us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 02:52:52.391149 (+ 158us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:52:52.392003 (+ 854us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:52:52.393762 (+ 1759us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:52:52.394605 (+ 843us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:52:52.394744 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:52:52.394888 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0811 02:52:52.396886 (+ 1998us) server_negotiation.cc:299] Negotiation successful
0811 02:52:52.396980 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":316,"thread_start_us":114,"threads_started":1}
I20260811 02:52:52.399085 25184 tablet_service.cc:1467] Processing CreateTablet for tablet 591961c806d844a88ae683eb19d3e2d0 (DEFAULT_TABLE table=test-table [id=6c92b1d884dd49a8b29fd51896d0eab1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 02:52:52.399174 25031 tablet_service.cc:1467] Processing CreateTablet for tablet 591961c806d844a88ae683eb19d3e2d0 (DEFAULT_TABLE table=test-table [id=6c92b1d884dd49a8b29fd51896d0eab1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 02:52:52.399675 25344 tablet_service.cc:1467] Processing CreateTablet for tablet 591961c806d844a88ae683eb19d3e2d0 (DEFAULT_TABLE table=test-table [id=6c92b1d884dd49a8b29fd51896d0eab1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 02:52:52.400152 25184 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 591961c806d844a88ae683eb19d3e2d0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 02:52:52.400261 25031 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 591961c806d844a88ae683eb19d3e2d0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 02:52:52.400651 25344 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 591961c806d844a88ae683eb19d3e2d0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 02:52:52.406415 25440 tablet_bootstrap.cc:492] T 591961c806d844a88ae683eb19d3e2d0 P 329fba0a0c4f4c639f4a8bb2449ccbd3: Bootstrap starting.
I20260811 02:52:52.407997 25441 tablet_bootstrap.cc:492] T 591961c806d844a88ae683eb19d3e2d0 P 3390c91c03de4b419bc50509131eacfa: Bootstrap starting.
I20260811 02:52:52.408445 25442 tablet_bootstrap.cc:492] T 591961c806d844a88ae683eb19d3e2d0 P 3cf081e80f20400bbf9fd1eaa41f9b95: Bootstrap starting.
I20260811 02:52:52.409468 25440 tablet_bootstrap.cc:654] T 591961c806d844a88ae683eb19d3e2d0 P 329fba0a0c4f4c639f4a8bb2449ccbd3: Neither blocks nor log segments found. Creating new log.
I20260811 02:52:52.410219 25440 log.cc:824] T 591961c806d844a88ae683eb19d3e2d0 P 329fba0a0c4f4c639f4a8bb2449ccbd3: Log is configured to *not* fsync() on all Append() calls
I20260811 02:52:52.410804 25441 tablet_bootstrap.cc:654] T 591961c806d844a88ae683eb19d3e2d0 P 3390c91c03de4b419bc50509131eacfa: Neither blocks nor log segments found. Creating new log.
I20260811 02:52:52.411278 25442 tablet_bootstrap.cc:654] T 591961c806d844a88ae683eb19d3e2d0 P 3cf081e80f20400bbf9fd1eaa41f9b95: Neither blocks nor log segments found. Creating new log.
I20260811 02:52:52.411824 25441 log.cc:824] T 591961c806d844a88ae683eb19d3e2d0 P 3390c91c03de4b419bc50509131eacfa: Log is configured to *not* fsync() on all Append() calls
I20260811 02:52:52.412171 25440 tablet_bootstrap.cc:492] T 591961c806d844a88ae683eb19d3e2d0 P 329fba0a0c4f4c639f4a8bb2449ccbd3: No bootstrap required, opened a new log
I20260811 02:52:52.412393 25440 ts_tablet_manager.cc:1397] T 591961c806d844a88ae683eb19d3e2d0 P 329fba0a0c4f4c639f4a8bb2449ccbd3: Time spent bootstrapping tablet: real 0.006s user 0.001s sys 0.003s
I20260811 02:52:52.412714 25442 log.cc:824] T 591961c806d844a88ae683eb19d3e2d0 P 3cf081e80f20400bbf9fd1eaa41f9b95: Log is configured to *not* fsync() on all Append() calls
I20260811 02:52:52.414186 25441 tablet_bootstrap.cc:492] T 591961c806d844a88ae683eb19d3e2d0 P 3390c91c03de4b419bc50509131eacfa: No bootstrap required, opened a new log
I20260811 02:52:52.414453 25441 ts_tablet_manager.cc:1397] T 591961c806d844a88ae683eb19d3e2d0 P 3390c91c03de4b419bc50509131eacfa: Time spent bootstrapping tablet: real 0.007s user 0.005s sys 0.000s
I20260811 02:52:52.414800 25442 tablet_bootstrap.cc:492] T 591961c806d844a88ae683eb19d3e2d0 P 3cf081e80f20400bbf9fd1eaa41f9b95: No bootstrap required, opened a new log
I20260811 02:52:52.414987 25442 ts_tablet_manager.cc:1397] T 591961c806d844a88ae683eb19d3e2d0 P 3cf081e80f20400bbf9fd1eaa41f9b95: Time spent bootstrapping tablet: real 0.007s user 0.004s sys 0.000s
I20260811 02:52:52.416038 25440 raft_consensus.cc:348] T 591961c806d844a88ae683eb19d3e2d0 P 329fba0a0c4f4c639f4a8bb2449ccbd3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3390c91c03de4b419bc50509131eacfa" member_type: VOTER last_known_addr { host: "127.12.138.193" port: 38729 } } peers { permanent_uuid: "3cf081e80f20400bbf9fd1eaa41f9b95" member_type: VOTER last_known_addr { host: "127.12.138.195" port: 34879 } } peers { permanent_uuid: "329fba0a0c4f4c639f4a8bb2449ccbd3" member_type: VOTER last_known_addr { host: "127.12.138.194" port: 45375 } }
I20260811 02:52:52.416285 25440 raft_consensus.cc:374] T 591961c806d844a88ae683eb19d3e2d0 P 329fba0a0c4f4c639f4a8bb2449ccbd3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 02:52:52.416361 25440 raft_consensus.cc:729] T 591961c806d844a88ae683eb19d3e2d0 P 329fba0a0c4f4c639f4a8bb2449ccbd3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 329fba0a0c4f4c639f4a8bb2449ccbd3, State: Initialized, Role: FOLLOWER
I20260811 02:52:52.416839 25440 consensus_queue.cc:260] T 591961c806d844a88ae683eb19d3e2d0 P 329fba0a0c4f4c639f4a8bb2449ccbd3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3390c91c03de4b419bc50509131eacfa" member_type: VOTER last_known_addr { host: "127.12.138.193" port: 38729 } } peers { permanent_uuid: "3cf081e80f20400bbf9fd1eaa41f9b95" member_type: VOTER last_known_addr { host: "127.12.138.195" port: 34879 } } peers { permanent_uuid: "329fba0a0c4f4c639f4a8bb2449ccbd3" member_type: VOTER last_known_addr { host: "127.12.138.194" port: 45375 } }
I20260811 02:52:52.417553 25263 heartbeater.cc:503] Master 127.12.138.254:40431 was elected leader, sending a full tablet report...
I20260811 02:52:52.419106 25440 ts_tablet_manager.cc:1428] T 591961c806d844a88ae683eb19d3e2d0 P 329fba0a0c4f4c639f4a8bb2449ccbd3: Time spent starting tablet: real 0.007s user 0.005s sys 0.002s
I20260811 02:52:52.419138 25442 raft_consensus.cc:348] T 591961c806d844a88ae683eb19d3e2d0 P 3cf081e80f20400bbf9fd1eaa41f9b95 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3390c91c03de4b419bc50509131eacfa" member_type: VOTER last_known_addr { host: "127.12.138.193" port: 38729 } } peers { permanent_uuid: "3cf081e80f20400bbf9fd1eaa41f9b95" member_type: VOTER last_known_addr { host: "127.12.138.195" port: 34879 } } peers { permanent_uuid: "329fba0a0c4f4c639f4a8bb2449ccbd3" member_type: VOTER last_known_addr { host: "127.12.138.194" port: 45375 } }
I20260811 02:52:52.419056 25441 raft_consensus.cc:348] T 591961c806d844a88ae683eb19d3e2d0 P 3390c91c03de4b419bc50509131eacfa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3390c91c03de4b419bc50509131eacfa" member_type: VOTER last_known_addr { host: "127.12.138.193" port: 38729 } } peers { permanent_uuid: "3cf081e80f20400bbf9fd1eaa41f9b95" member_type: VOTER last_known_addr { host: "127.12.138.195" port: 34879 } } peers { permanent_uuid: "329fba0a0c4f4c639f4a8bb2449ccbd3" member_type: VOTER last_known_addr { host: "127.12.138.194" port: 45375 } }
I20260811 02:52:52.419374 25441 raft_consensus.cc:374] T 591961c806d844a88ae683eb19d3e2d0 P 3390c91c03de4b419bc50509131eacfa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 02:52:52.419416 25442 raft_consensus.cc:374] T 591961c806d844a88ae683eb19d3e2d0 P 3cf081e80f20400bbf9fd1eaa41f9b95 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 02:52:52.419476 25441 raft_consensus.cc:729] T 591961c806d844a88ae683eb19d3e2d0 P 3390c91c03de4b419bc50509131eacfa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3390c91c03de4b419bc50509131eacfa, State: Initialized, Role: FOLLOWER
I20260811 02:52:52.419536 25442 raft_consensus.cc:729] T 591961c806d844a88ae683eb19d3e2d0 P 3cf081e80f20400bbf9fd1eaa41f9b95 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3cf081e80f20400bbf9fd1eaa41f9b95, State: Initialized, Role: FOLLOWER
I20260811 02:52:52.419988 25441 consensus_queue.cc:260] T 591961c806d844a88ae683eb19d3e2d0 P 3390c91c03de4b419bc50509131eacfa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3390c91c03de4b419bc50509131eacfa" member_type: VOTER last_known_addr { host: "127.12.138.193" port: 38729 } } peers { permanent_uuid: "3cf081e80f20400bbf9fd1eaa41f9b95" member_type: VOTER last_known_addr { host: "127.12.138.195" port: 34879 } } peers { permanent_uuid: "329fba0a0c4f4c639f4a8bb2449ccbd3" member_type: VOTER last_known_addr { host: "127.12.138.194" port: 45375 } }
I20260811 02:52:52.419987 25442 consensus_queue.cc:260] T 591961c806d844a88ae683eb19d3e2d0 P 3cf081e80f20400bbf9fd1eaa41f9b95 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3390c91c03de4b419bc50509131eacfa" member_type: VOTER last_known_addr { host: "127.12.138.193" port: 38729 } } peers { permanent_uuid: "3cf081e80f20400bbf9fd1eaa41f9b95" member_type: VOTER last_known_addr { host: "127.12.138.195" port: 34879 } } peers { permanent_uuid: "329fba0a0c4f4c639f4a8bb2449ccbd3" member_type: VOTER last_known_addr { host: "127.12.138.194" port: 45375 } }
I20260811 02:52:52.420939 25107 heartbeater.cc:503] Master 127.12.138.254:40431 was elected leader, sending a full tablet report...
I20260811 02:52:52.421021 25441 ts_tablet_manager.cc:1428] T 591961c806d844a88ae683eb19d3e2d0 P 3390c91c03de4b419bc50509131eacfa: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260811 02:52:52.421021 25442 ts_tablet_manager.cc:1428] T 591961c806d844a88ae683eb19d3e2d0 P 3cf081e80f20400bbf9fd1eaa41f9b95: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260811 02:52:52.422238 25420 heartbeater.cc:503] Master 127.12.138.254:40431 was elected leader, sending a full tablet report...
I20260811 02:52:52.488449 25449 raft_consensus.cc:482] T 591961c806d844a88ae683eb19d3e2d0 P 329fba0a0c4f4c639f4a8bb2449ccbd3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 02:52:52.488685 25449 raft_consensus.cc:504] T 591961c806d844a88ae683eb19d3e2d0 P 329fba0a0c4f4c639f4a8bb2449ccbd3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3390c91c03de4b419bc50509131eacfa" member_type: VOTER last_known_addr { host: "127.12.138.193" port: 38729 } } peers { permanent_uuid: "3cf081e80f20400bbf9fd1eaa41f9b95" member_type: VOTER last_known_addr { host: "127.12.138.195" port: 34879 } } peers { permanent_uuid: "329fba0a0c4f4c639f4a8bb2449ccbd3" member_type: VOTER last_known_addr { host: "127.12.138.194" port: 45375 } }
I20260811 02:52:52.489960 25449 leader_election.cc:288] T 591961c806d844a88ae683eb19d3e2d0 P 329fba0a0c4f4c639f4a8bb2449ccbd3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3390c91c03de4b419bc50509131eacfa (127.12.138.193:38729), 3cf081e80f20400bbf9fd1eaa41f9b95 (127.12.138.195:34879)
I20260811 02:52:52.495061 25438 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:52:52.490373 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.138.194:54477 (local address 127.12.138.195:34879)
0811 02:52:52.490514 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0811 02:52:52.490520 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 02:52:52.490808 (+ 288us) server_negotiation.cc:407] Connection header received
0811 02:52:52.490944 (+ 136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:52:52.490950 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:52:52.491000 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:52:52.491068 (+ 68us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 02:52:52.491364 (+ 296us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:52:52.492059 (+ 695us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:52:52.493665 (+ 1606us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:52:52.494347 (+ 682us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:52:52.494462 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:52:52.494517 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0811 02:52:52.494857 (+ 340us) server_negotiation.cc:299] Negotiation successful
0811 02:52:52.494927 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260811 02:52:52.495061 25435 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:52:52.490493 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.138.194:60953 (local address 127.12.138.193:38729)
0811 02:52:52.490632 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0811 02:52:52.490635 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 02:52:52.490647 (+ 12us) server_negotiation.cc:407] Connection header received
0811 02:52:52.490694 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:52:52.490696 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:52:52.490731 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:52:52.490838 (+ 107us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 02:52:52.491146 (+ 308us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:52:52.491926 (+ 780us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:52:52.493678 (+ 1752us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:52:52.494332 (+ 654us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:52:52.494462 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:52:52.494517 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0811 02:52:52.494857 (+ 340us) server_negotiation.cc:299] Negotiation successful
0811 02:52:52.494933 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260811 02:52:52.495097 25265 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:52:52.490185 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.138.193:38729 (local address 127.12.138.194:60953)
0811 02:52:52.490346 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0811 02:52:52.490373 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0811 02:52:52.490495 (+ 122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:52:52.490861 (+ 366us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:52:52.490867 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:52:52.490886 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 02:52:52.491011 (+ 125us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:52:52.491015 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:52:52.492036 (+ 1021us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:52:52.492039 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:52:52.493526 (+ 1487us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:52:52.493538 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:52:52.494469 (+ 931us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:52:52.494473 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:52:52.494651 (+ 178us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:52:52.494721 (+ 70us) client_negotiation.cc:769] Sending connection context
0811 02:52:52.494810 (+ 89us) client_negotiation.cc:240] Negotiation successful
0811 02:52:52.494926 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":42,"mutex_wait_us":159}
I20260811 02:52:52.495162 25454 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:52:52.490193 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.138.195:34879 (local address 127.12.138.194:54477)
0811 02:52:52.490698 (+ 505us) negotiation.cc:107] Waiting for socket to connect
0811 02:52:52.490710 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 02:52:52.490831 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:52:52.491102 (+ 271us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:52:52.491105 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:52:52.491125 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 02:52:52.491220 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:52:52.491224 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:52:52.492151 (+ 927us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:52:52.492154 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:52:52.493526 (+ 1372us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:52:52.493538 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:52:52.494469 (+ 931us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:52:52.494473 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:52:52.494618 (+ 145us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:52:52.494725 (+ 107us) client_negotiation.cc:769] Sending connection context
0811 02:52:52.494810 (+ 85us) client_negotiation.cc:240] Negotiation successful
0811 02:52:52.494925 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":452,"mutex_wait_us":269,"thread_start_us":134,"threads_started":1}
I20260811 02:52:52.495908 25057 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "591961c806d844a88ae683eb19d3e2d0" candidate_uuid: "329fba0a0c4f4c639f4a8bb2449ccbd3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3390c91c03de4b419bc50509131eacfa" is_pre_election: true
I20260811 02:52:52.495985 25370 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "591961c806d844a88ae683eb19d3e2d0" candidate_uuid: "329fba0a0c4f4c639f4a8bb2449ccbd3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3cf081e80f20400bbf9fd1eaa41f9b95" is_pre_election: true
I20260811 02:52:52.496277 25370 raft_consensus.cc:2454] T 591961c806d844a88ae683eb19d3e2d0 P 3cf081e80f20400bbf9fd1eaa41f9b95 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 329fba0a0c4f4c639f4a8bb2449ccbd3 in term 0.
I20260811 02:52:52.496279 25057 raft_consensus.cc:2454] T 591961c806d844a88ae683eb19d3e2d0 P 3390c91c03de4b419bc50509131eacfa [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 329fba0a0c4f4c639f4a8bb2449ccbd3 in term 0.
I20260811 02:52:52.496855 25134 leader_election.cc:302] T 591961c806d844a88ae683eb19d3e2d0 P 329fba0a0c4f4c639f4a8bb2449ccbd3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 329fba0a0c4f4c639f4a8bb2449ccbd3, 3390c91c03de4b419bc50509131eacfa, 3cf081e80f20400bbf9fd1eaa41f9b95; no voters:
I20260811 02:52:52.497417 25449 raft_consensus.cc:2781] T 591961c806d844a88ae683eb19d3e2d0 P 329fba0a0c4f4c639f4a8bb2449ccbd3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 02:52:52.497527 25449 raft_consensus.cc:482] T 591961c806d844a88ae683eb19d3e2d0 P 329fba0a0c4f4c639f4a8bb2449ccbd3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 02:52:52.497622 25449 raft_consensus.cc:3037] T 591961c806d844a88ae683eb19d3e2d0 P 329fba0a0c4f4c639f4a8bb2449ccbd3 [term 0 FOLLOWER]: Advancing to term 1
I20260811 02:52:52.498740 25449 raft_consensus.cc:504] T 591961c806d844a88ae683eb19d3e2d0 P 329fba0a0c4f4c639f4a8bb2449ccbd3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3390c91c03de4b419bc50509131eacfa" member_type: VOTER last_known_addr { host: "127.12.138.193" port: 38729 } } peers { permanent_uuid: "3cf081e80f20400bbf9fd1eaa41f9b95" member_type: VOTER last_known_addr { host: "127.12.138.195" port: 34879 } } peers { permanent_uuid: "329fba0a0c4f4c639f4a8bb2449ccbd3" member_type: VOTER last_known_addr { host: "127.12.138.194" port: 45375 } }
I20260811 02:52:52.499276 25449 leader_election.cc:288] T 591961c806d844a88ae683eb19d3e2d0 P 329fba0a0c4f4c639f4a8bb2449ccbd3 [CANDIDATE]: Term 1 election: Requested vote from peers 3390c91c03de4b419bc50509131eacfa (127.12.138.193:38729), 3cf081e80f20400bbf9fd1eaa41f9b95 (127.12.138.195:34879)
I20260811 02:52:52.499676 25370 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "591961c806d844a88ae683eb19d3e2d0" candidate_uuid: "329fba0a0c4f4c639f4a8bb2449ccbd3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3cf081e80f20400bbf9fd1eaa41f9b95"
I20260811 02:52:52.499691 25057 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "591961c806d844a88ae683eb19d3e2d0" candidate_uuid: "329fba0a0c4f4c639f4a8bb2449ccbd3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3390c91c03de4b419bc50509131eacfa"
I20260811 02:52:52.499796 25370 raft_consensus.cc:3037] T 591961c806d844a88ae683eb19d3e2d0 P 3cf081e80f20400bbf9fd1eaa41f9b95 [term 0 FOLLOWER]: Advancing to term 1
I20260811 02:52:52.499850 25057 raft_consensus.cc:3037] T 591961c806d844a88ae683eb19d3e2d0 P 3390c91c03de4b419bc50509131eacfa [term 0 FOLLOWER]: Advancing to term 1
I20260811 02:52:52.501029 25370 raft_consensus.cc:2454] T 591961c806d844a88ae683eb19d3e2d0 P 3cf081e80f20400bbf9fd1eaa41f9b95 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 329fba0a0c4f4c639f4a8bb2449ccbd3 in term 1.
I20260811 02:52:52.501056 25057 raft_consensus.cc:2454] T 591961c806d844a88ae683eb19d3e2d0 P 3390c91c03de4b419bc50509131eacfa [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 329fba0a0c4f4c639f4a8bb2449ccbd3 in term 1.
I20260811 02:52:52.501520 25133 leader_election.cc:302] T 591961c806d844a88ae683eb19d3e2d0 P 329fba0a0c4f4c639f4a8bb2449ccbd3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 329fba0a0c4f4c639f4a8bb2449ccbd3, 3390c91c03de4b419bc50509131eacfa, 3cf081e80f20400bbf9fd1eaa41f9b95; no voters:
I20260811 02:52:52.501986 25449 raft_consensus.cc:2781] T 591961c806d844a88ae683eb19d3e2d0 P 329fba0a0c4f4c639f4a8bb2449ccbd3 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 02:52:52.502276 25449 raft_consensus.cc:686] T 591961c806d844a88ae683eb19d3e2d0 P 329fba0a0c4f4c639f4a8bb2449ccbd3 [term 1 LEADER]: Becoming Leader. State: Replica: 329fba0a0c4f4c639f4a8bb2449ccbd3, State: Running, Role: LEADER
I20260811 02:52:52.502715 25449 consensus_queue.cc:237] T 591961c806d844a88ae683eb19d3e2d0 P 329fba0a0c4f4c639f4a8bb2449ccbd3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3390c91c03de4b419bc50509131eacfa" member_type: VOTER last_known_addr { host: "127.12.138.193" port: 38729 } } peers { permanent_uuid: "3cf081e80f20400bbf9fd1eaa41f9b95" member_type: VOTER last_known_addr { host: "127.12.138.195" port: 34879 } } peers { permanent_uuid: "329fba0a0c4f4c639f4a8bb2449ccbd3" member_type: VOTER last_known_addr { host: "127.12.138.194" port: 45375 } }
I20260811 02:52:52.506359 24889 catalog_manager.cc:5495] T 591961c806d844a88ae683eb19d3e2d0 P 329fba0a0c4f4c639f4a8bb2449ccbd3 reported cstate change: term changed from 0 to 1, leader changed from <none> to 329fba0a0c4f4c639f4a8bb2449ccbd3 (127.12.138.194). New cstate: current_term: 1 leader_uuid: "329fba0a0c4f4c639f4a8bb2449ccbd3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3390c91c03de4b419bc50509131eacfa" member_type: VOTER last_known_addr { host: "127.12.138.193" port: 38729 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3cf081e80f20400bbf9fd1eaa41f9b95" member_type: VOTER last_known_addr { host: "127.12.138.195" port: 34879 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "329fba0a0c4f4c639f4a8bb2449ccbd3" member_type: VOTER last_known_addr { host: "127.12.138.194" port: 45375 } health_report { overall_health: HEALTHY } } }
I20260811 02:52:52.522070 25439 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:52:52.518125 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54994 (local address 127.12.138.194:45375)
0811 02:52:52.518256 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0811 02:52:52.518260 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 02:52:52.518274 (+ 14us) server_negotiation.cc:407] Connection header received
0811 02:52:52.518350 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:52:52.518352 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:52:52.518408 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:52:52.518506 (+ 98us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 02:52:52.518894 (+ 388us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:52:52.519461 (+ 567us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:52:52.520513 (+ 1052us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:52:52.520728 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:52:52.520814 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:52:52.521124 (+ 310us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 02:52:52.521665 (+ 541us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 02:52:52.521779 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0811 02:52:52.521905 (+ 126us) server_negotiation.cc:299] Negotiation successful
0811 02:52:52.521961 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260811 02:52:52.528461 25370 raft_consensus.cc:1261] T 591961c806d844a88ae683eb19d3e2d0 P 3cf081e80f20400bbf9fd1eaa41f9b95 [term 1 FOLLOWER]: Refusing update from remote peer 329fba0a0c4f4c639f4a8bb2449ccbd3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 02:52:52.528461 25057 raft_consensus.cc:1261] T 591961c806d844a88ae683eb19d3e2d0 P 3390c91c03de4b419bc50509131eacfa [term 1 FOLLOWER]: Refusing update from remote peer 329fba0a0c4f4c639f4a8bb2449ccbd3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 02:52:52.529353 25455 consensus_queue.cc:1035] T 591961c806d844a88ae683eb19d3e2d0 P 329fba0a0c4f4c639f4a8bb2449ccbd3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3390c91c03de4b419bc50509131eacfa" member_type: VOTER last_known_addr { host: "127.12.138.193" port: 38729 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 02:52:52.529598 25449 consensus_queue.cc:1035] T 591961c806d844a88ae683eb19d3e2d0 P 329fba0a0c4f4c639f4a8bb2449ccbd3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3cf081e80f20400bbf9fd1eaa41f9b95" member_type: VOTER last_known_addr { host: "127.12.138.195" port: 34879 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 02:52:52.538405 25461 mvcc.cc:205] Tried to move back new op lower bound from 7317163100268699648 to 7317163100173860864. Current Snapshot: MvccSnapshot[applied={T|T < 7317163100268699648}]
I20260811 02:52:52.546406 24887 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55454:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 02:52:52.546567 24887 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55454:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 02:52:52.548954 24887 catalog_manager.cc:5782] T 00000000000000000000000000000000 P c01c5e4e2fe0402e953959a5d5dccfa9: Sending DeleteTablet for 3 replicas of tablet 591961c806d844a88ae683eb19d3e2d0
I20260811 02:52:52.549983 25031 tablet_service.cc:1514] Processing DeleteTablet for tablet 591961c806d844a88ae683eb19d3e2d0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 02:52:52 UTC) from {username='kudu', principal='kudu/127.12.138.254@KRBTEST.COM'} at 127.0.0.1:40920
I20260811 02:52:52.550069 25184 tablet_service.cc:1514] Processing DeleteTablet for tablet 591961c806d844a88ae683eb19d3e2d0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 02:52:52 UTC) from {username='kudu', principal='kudu/127.12.138.254@KRBTEST.COM'} at 127.0.0.1:54980
I20260811 02:52:52.550096 25344 tablet_service.cc:1514] Processing DeleteTablet for tablet 591961c806d844a88ae683eb19d3e2d0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 02:52:52 UTC) from {username='kudu', principal='kudu/127.12.138.254@KRBTEST.COM'} at 127.0.0.1:54356
I20260811 02:52:52.550644 25473 tablet_replica.cc:330] T 591961c806d844a88ae683eb19d3e2d0 P 3cf081e80f20400bbf9fd1eaa41f9b95: stopping tablet replica
I20260811 02:52:52.550719 25470 tablet_replica.cc:330] T 591961c806d844a88ae683eb19d3e2d0 P 329fba0a0c4f4c639f4a8bb2449ccbd3: stopping tablet replica
I20260811 02:52:52.551031 25471 tablet_replica.cc:330] T 591961c806d844a88ae683eb19d3e2d0 P 3390c91c03de4b419bc50509131eacfa: stopping tablet replica
I20260811 02:52:52.551044 25473 raft_consensus.cc:2229] T 591961c806d844a88ae683eb19d3e2d0 P 3cf081e80f20400bbf9fd1eaa41f9b95 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 02:52:52.551345 12843 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDawEwS/build/debug/bin/kudu with pid 24960
I20260811 02:52:52.551923 25473 raft_consensus.cc:2258] T 591961c806d844a88ae683eb19d3e2d0 P 3cf081e80f20400bbf9fd1eaa41f9b95 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 02:52:52.551401 25470 raft_consensus.cc:2229] T 591961c806d844a88ae683eb19d3e2d0 P 329fba0a0c4f4c639f4a8bb2449ccbd3 [term 1 LEADER]: Raft consensus shutting down.
I20260811 02:52:52.555500 25470 raft_consensus.cc:2258] T 591961c806d844a88ae683eb19d3e2d0 P 329fba0a0c4f4c639f4a8bb2449ccbd3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 02:52:52.557578 25470 ts_tablet_manager.cc:1905] T 591961c806d844a88ae683eb19d3e2d0 P 329fba0a0c4f4c639f4a8bb2449ccbd3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 02:52:52.557727 25473 ts_tablet_manager.cc:1905] T 591961c806d844a88ae683eb19d3e2d0 P 3cf081e80f20400bbf9fd1eaa41f9b95: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 02:52:52.560436 25470 ts_tablet_manager.cc:1918] T 591961c806d844a88ae683eb19d3e2d0 P 329fba0a0c4f4c639f4a8bb2449ccbd3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 02:52:52.560587 25470 log.cc:1196] T 591961c806d844a88ae683eb19d3e2d0 P 329fba0a0c4f4c639f4a8bb2449ccbd3: Deleting WAL directory at /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786416707705494-12843-0/minicluster-data/ts-1/wal/wals/591961c806d844a88ae683eb19d3e2d0
I20260811 02:52:52.560968 25470 ts_tablet_manager.cc:1939] T 591961c806d844a88ae683eb19d3e2d0 P 329fba0a0c4f4c639f4a8bb2449ccbd3: Deleting consensus metadata
I20260811 02:52:52.561172 25473 ts_tablet_manager.cc:1918] T 591961c806d844a88ae683eb19d3e2d0 P 3cf081e80f20400bbf9fd1eaa41f9b95: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 02:52:52.561321 25473 log.cc:1196] T 591961c806d844a88ae683eb19d3e2d0 P 3cf081e80f20400bbf9fd1eaa41f9b95: Deleting WAL directory at /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786416707705494-12843-0/minicluster-data/ts-2/wal/wals/591961c806d844a88ae683eb19d3e2d0
I20260811 02:52:52.561702 25473 ts_tablet_manager.cc:1939] T 591961c806d844a88ae683eb19d3e2d0 P 3cf081e80f20400bbf9fd1eaa41f9b95: Deleting consensus metadata
I20260811 02:52:52.562283 24871 catalog_manager.cc:4841] TS 329fba0a0c4f4c639f4a8bb2449ccbd3 (127.12.138.194:45375): tablet 591961c806d844a88ae683eb19d3e2d0 (table test-table [id=6c92b1d884dd49a8b29fd51896d0eab1]) successfully deleted
W20260811 02:52:52.562778 24872 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.12.138.193:38729 (error 108)
I20260811 02:52:52.562974 24873 catalog_manager.cc:4841] TS 3cf081e80f20400bbf9fd1eaa41f9b95 (127.12.138.195:34879): tablet 591961c806d844a88ae683eb19d3e2d0 (table test-table [id=6c92b1d884dd49a8b29fd51896d0eab1]) successfully deleted
I20260811 02:52:52.563261 12843 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDawEwS/build/debug/bin/kudu with pid 25113
I20260811 02:52:52.566059 25436 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:52:52.565681 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.138.193:38729 (local address 127.0.0.1:40934)
0811 02:52:52.565868 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0811 02:52:52.565947 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.138.193:38729: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":81}
W20260811 02:52:52.567214 24872 catalog_manager.cc:4584] TS 3390c91c03de4b419bc50509131eacfa (127.12.138.193:38729): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 591961c806d844a88ae683eb19d3e2d0: Network error: Client connection negotiation failed: client connection to 127.12.138.193:38729: connect: Connection refused (error 111)
I20260811 02:52:52.572685 12843 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDawEwS/build/debug/bin/kudu with pid 25269
I20260811 02:52:52.581351 12843 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDawEwS/build/debug/bin/kudu with pid 24851
2026-08-11T02:52:52Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationSecureCluster (2544 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegration
Loading random data
Initializing database '/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786416707705494-12843-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 02:52:52 dist-test-slave-9gcw krb5kdc[25478](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 02:52:52 dist-test-slave-9gcw krb5kdc[25478](info): set up 2 sockets
Aug 11 02:52:52 dist-test-slave-9gcw krb5kdc[25478](info): commencing operation
krb5kdc: starting...
W20260811 02:52:53.726265 12843 mini_kdc.cc:121] Time spent starting KDC: real 1.118s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786416707705494-12843-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786416707705494-12843-0/krb5kdc/test-user.keytab.
Aug 11 02:52:53 dist-test-slave-9gcw krb5kdc[25478](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416773, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T02:52:53Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T02:52:53Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.12.138.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.12.138.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.12.138.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786416707705494-12843-0/krb5kdc/rangeradmin_127.12.138.212@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.12.138.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786416707705494-12843-0/krb5kdc/rangeradmin_127.12.138.212@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.12.138.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.12.138.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.12.138.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786416707705494-12843-0/krb5kdc/rangerlookup_127.12.138.212@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.12.138.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786416707705494-12843-0/krb5kdc/rangerlookup_127.12.138.212@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.12.138.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.12.138.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.12.138.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786416707705494-12843-0/krb5kdc/HTTP_127.12.138.212@KRBTEST.COM.keytab.
Entry for principal HTTP/127.12.138.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786416707705494-12843-0/krb5kdc/HTTP_127.12.138.212@KRBTEST.COM.keytab.
I20260811 02:52:53.933353 12843 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786416707705494-12843-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskDawEwS/build/debug/bin/postgres/pg_ctl -D /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786416707705494-12843-0/minicluster-data/postgres -l logfile start
2026-08-11 02:52:56.722 UTC [25517] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-08-11 02:52:56.722 UTC [25517] LOG: listening on IPv4 address "127.12.138.212", port 60981
2026-08-11 02:52:56.726 UTC [25517] LOG: listening on Unix socket "/tmp/.s.PGSQL.60981"
2026-08-11 02:52:56.749 UTC [25521] LOG: database system was shut down at 2026-08-11 02:52:55 UTC
2026-08-11 02:52:56.753 UTC [25517] LOG: database system is ready to accept connections
I20260811 02:52:57.813458 12843 mini_postgres.cc:96] Postgres bound to 60981
2026-08-11 02:52:57.821 UTC [25531] FATAL: database "slave" does not exist
127.12.138.212:60981 - accepting connections
I20260811 02:52:57.822623 12843 mini_ranger.cc:161] Starting Ranger...
I20260811 02:52:57.839999 12843 mini_ranger.cc:84] Created miniranger Postgres user
I20260811 02:52:58.388628 12843 mini_ranger.cc:87] Created ranger Postgres database
I20260811 02:52:58.388728 12843 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786416707705494-12843-0/minicluster-data/ranger-admin
2026-08-11 02:52:58,707 [I] DB FLAVOR :POSTGRES
2026-08-11 02:52:58,707 [I] --------- Verifying Ranger DB connection ---------
2026-08-11 02:52:58,707 [I] Checking connection..
2026-08-11 02:52:58,708 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDawEwS/build/debug/bin/postgresql.jar:/tmp/dist-test-taskDawEwS/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.12.138.212:60981/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-08-11 02:52:58,970 [I] Checking connection passed.
2026-08-11 02:52:58,970 [I] --------- Verifying version history table ---------
2026-08-11 02:52:58,970 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDawEwS/build/debug/bin/postgresql.jar:/tmp/dist-test-taskDawEwS/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.12.138.212:60981/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-11 02:52:59,236 [I] Table x_db_version_h does not exist in database ranger
2026-08-11 02:52:59,237 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDawEwS/build/debug/bin/postgresql.jar:/tmp/dist-test-taskDawEwS/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.12.138.212:60981/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-11 02:52:59,487 [I] Table x_db_version_h does not exist in database ranger
2026-08-11 02:52:59,487 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-08-11 02:52:59,487 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDawEwS/build/debug/bin/postgresql.jar:/tmp/dist-test-taskDawEwS/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.12.138.212:60981/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskDawEwS/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-08-11 02:52:59.760 UTC [25611] WARNING: there is no transaction in progress
2026-08-11 02:52:59,767 [I] create_dbversion_catalog.sql file imported successfully
2026-08-11 02:52:59,767 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDawEwS/build/debug/bin/postgresql.jar:/tmp/dist-test-taskDawEwS/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.12.138.212:60981/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-11 02:53:00,044 [I] Table x_db_version_h already exists in database 'ranger'
2026-08-11 02:53:00,044 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-11 02:53:00,044 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDawEwS/build/debug/bin/postgresql.jar:/tmp/dist-test-taskDawEwS/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.12.138.212:60981/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-08-11 02:53:00,298 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDawEwS/build/debug/bin/postgresql.jar:/tmp/dist-test-taskDawEwS/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.12.138.212:60981/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-08-11 02:53:00,553 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDawEwS/build/debug/bin/postgresql.jar:/tmp/dist-test-taskDawEwS/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.12.138.212:60981/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-9gcw.c.gcp-upstream.internal','N') ;"
2026-08-11 02:53:00,793 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDawEwS/build/debug/bin/postgresql.jar:/tmp/dist-test-taskDawEwS/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.12.138.212:60981/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-11 02:53:01,067 [I] Table x_portal_user does not exist in database ranger
2026-08-11 02:53:01,068 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDawEwS/build/debug/bin/postgresql.jar:/tmp/dist-test-taskDawEwS/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.12.138.212:60981/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-11 02:53:01,358 [I] Table x_policy_ref_group does not exist in database ranger
2026-08-11 02:53:01,359 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDawEwS/build/debug/bin/postgresql.jar:/tmp/dist-test-taskDawEwS/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.12.138.212:60981/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-11 02:53:01,641 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDawEwS/build/debug/bin/postgresql.jar:/tmp/dist-test-taskDawEwS/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.12.138.212:60981/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-11 02:53:01,912 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-08-11 02:53:01,913 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDawEwS/build/debug/bin/postgresql.jar:/tmp/dist-test-taskDawEwS/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.12.138.212:60981/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskDawEwS/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-08-11 02:53:02.980 UTC [25739] WARNING: there is no transaction in progress
2026-08-11 02:53:03.000 UTC [25739] WARNING: there is no transaction in progress
2026-08-11 02:53:03.020 UTC [25739] WARNING: there is no transaction in progress
2026-08-11 02:53:03.039 UTC [25739] WARNING: there is no transaction in progress
2026-08-11 02:53:03.059 UTC [25739] WARNING: there is no transaction in progress
2026-08-11 02:53:03.079 UTC [25739] WARNING: there is no transaction in progress
2026-08-11 02:53:03.144 UTC [25739] WARNING: there is no transaction in progress
2026-08-11 02:53:03.152 UTC [25739] WARNING: there is no transaction in progress
2026-08-11 02:53:03.169 UTC [25739] WARNING: there is no transaction in progress
2026-08-11 02:53:03.181 UTC [25739] WARNING: there is no transaction in progress
2026-08-11 02:53:03.194 UTC [25739] WARNING: there is no transaction in progress
2026-08-11 02:53:03.210 UTC [25739] WARNING: there is no transaction in progress
2026-08-11 02:53:03.221 UTC [25739] WARNING: there is no transaction in progress
2026-08-11 02:53:03.228 UTC [25739] WARNING: there is no transaction in progress
2026-08-11 02:53:03.236 UTC [25739] WARNING: there is no transaction in progress
2026-08-11 02:53:03.245 UTC [25739] WARNING: there is no transaction in progress
2026-08-11 02:53:04,033 [I] ranger_core_db_postgres.sql file imported successfully
2026-08-11 02:53:04,034 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDawEwS/build/debug/bin/postgresql.jar:/tmp/dist-test-taskDawEwS/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.12.138.212:60981/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-08-11 02:53:04,289 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-08-11 02:53:04,289 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDawEwS/build/debug/bin/postgresql.jar:/tmp/dist-test-taskDawEwS/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.12.138.212:60981/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-11 02:53:04,534 [I] Table x_portal_user already exists in database 'ranger'
2026-08-11 02:53:04,535 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDawEwS/build/debug/bin/postgresql.jar:/tmp/dist-test-taskDawEwS/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.12.138.212:60981/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-11 02:53:04,826 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-08-11 02:53:04,827 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDawEwS/build/debug/bin/postgresql.jar:/tmp/dist-test-taskDawEwS/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.12.138.212:60981/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-11 02:53:05,110 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDawEwS/build/debug/bin/postgresql.jar:/tmp/dist-test-taskDawEwS/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.12.138.212:60981/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-11 02:53:05,368 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDawEwS/build/debug/bin/postgresql.jar:/tmp/dist-test-taskDawEwS/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.12.138.212:60981/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-9gcw.c.gcp-upstream.internal';"
2026-08-11 02:53:05,619 [I] CORE_DB_SCHEMA import status has been updated
2026-08-11 02:53:05,620 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDawEwS/build/debug/bin/postgresql.jar:/tmp/dist-test-taskDawEwS/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.12.138.212:60981/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-08-11 02:53:05,900 [I] DB_PATCHES have already been applied
I20260811 02:53:05.906301 12843 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786416707705494-12843-0/minicluster-data/ranger-admin:/tmp/dist-test-taskDawEwS/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskDawEwS/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskDawEwS/build/debug/bin/postgresql.jar:/tmp/dist-test-taskDawEwS/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260811 02:53:05.906376 12843 mini_ranger.cc:193] Using host: 127.12.138.212
I20260811 02:53:05.911496 12843 mini_ranger.cc:227] Ranger admin URL: http://127.12.138.212:45355
Aug 11, 2026 2:53:06 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 11, 2026 2:53:06 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 11, 2026 2:53:06 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786416707705494-12843-0/minicluster-data/ranger-admin/ews/webapp
Aug 11, 2026 2:53:06 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786416707705494-12843-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Aug 11, 2026 2:53:06 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786416707705494-12843-0/minicluster-data/ranger-admin/ews/webapp] .....
Aug 11, 2026 2:53:06 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786416707705494-12843-0/minicluster-data/ranger-admin/ews/webapp].
Aug 11, 2026 2:53:06 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260811 02:53:07.162477 12843 mini_ranger.cc:160] Time spent starting Ranger: real 9.340s user 0.000s sys 0.015s
Aug 11, 2026 2:53:29 AM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Aug 11, 2026 2:53:29 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.UserREST
Aug 11, 2026 2:53:29 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Aug 11, 2026 2:53:29 AM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Aug 11, 2026 2:53:29 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Aug 11, 2026 2:53:29 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Aug 11, 2026 2:53:29 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Aug 11, 2026 2:53:29 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Aug 11, 2026 2:53:29 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Aug 11, 2026 2:53:29 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Aug 11, 2026 2:53:29 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Aug 11, 2026 2:53:29 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Aug 11, 2026 2:53:29 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Aug 11, 2026 2:53:29 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Aug 11, 2026 2:53:29 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Aug 11, 2026 2:53:29 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Aug 11, 2026 2:53:29 AM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Aug 11, 2026 2:53:30 AM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260811 02:53:30.681393 12843 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.12.138.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.12.138.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.12.138.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786416707705494-12843-0/krb5kdc/rangerkms_127.12.138.212@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.12.138.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786416707705494-12843-0/krb5kdc/rangerkms_127.12.138.212@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.12.138.212@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.12.138.212@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.12.138.212@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786416707705494-12843-0/krb5kdc/HTTP_127.12.138.212@KRBTEST.COM.keytab.
Entry for principal HTTP/127.12.138.212@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786416707705494-12843-0/krb5kdc/HTTP_127.12.138.212@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Aug 11 02:53:30 dist-test-slave-9gcw krb5kdc[25478](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416810, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260811 02:53:30.780973 12843 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260811 02:53:30.794103 12843 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260811 02:53:31.783634 12843 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260811 02:53:31.783763 12843 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786416707705494-12843-0/minicluster-data/ranger-kms
I20260811 02:53:31.783881 12843 mini_ranger_kms.cc:227] Using postgres at 127.12.138.212:60981
2026-08-11 02:53:32,074 [I] DB FLAVOR :POSTGRES
2026-08-11 02:53:32,075 [I] --------- Verifying Ranger DB connection ---------
2026-08-11 02:53:32,075 [I] Checking connection
2026-08-11 02:53:32,075 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDawEwS/build/debug/bin/postgresql.jar:/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786416707705494-12843-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.12.138.212:60981/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-08-11 02:53:32,390 [I] connection success
2026-08-11 02:53:32,390 [I] --------- Verifying Ranger DB tables ---------
2026-08-11 02:53:32,390 [I] Verifying table ranger_masterkey in database rangerkms
2026-08-11 02:53:32,390 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDawEwS/build/debug/bin/postgresql.jar:/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786416707705494-12843-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.12.138.212:60981/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-08-11 02:53:32,736 [I] Table ranger_masterkey does not exist in database rangerkms
2026-08-11 02:53:32,737 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-11 02:53:32,737 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-08-11 02:53:32,737 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDawEwS/build/debug/bin/postgresql.jar:/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786416707705494-12843-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.12.138.212:60981/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786416707705494-12843-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-08-11 02:53:33,095 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260811 02:53:33.628263 12843 mini_ranger_kms.cc:310] Created kms service
I20260811 02:53:33.867406 12843 mini_ranger_kms.cc:326] Created kudu user
I20260811 02:53:33.912997 12843 mini_ranger_kms.cc:343] Created rangerkms user
Aug 11 02:53:33 dist-test-slave-9gcw krb5kdc[25478](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416810, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.12.138.212@KRBTEST.COM
I20260811 02:53:34.321422 12843 mini_ranger_kms.cc:384] Added ranger policy
I20260811 02:53:34.321604 12843 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786416707705494-12843-0/minicluster-data/ranger-kms:/tmp/dist-test-taskDawEwS/build/debug/bin/postgresql.jar:/tmp/dist-test-taskDawEwS/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskDawEwS/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260811 02:53:34.321637 12843 mini_ranger_kms.cc:240] Using host: 127.12.138.212
I20260811 02:53:34.327153 12843 mini_ranger_kms.cc:276] Ranger KMS PID: 25957
I20260811 02:53:34.327329 12843 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.12.138.212:54539
Aug 11, 2026 2:53:35 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 11, 2026 2:53:35 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 11, 2026 2:53:35 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786416707705494-12843-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Aug 11, 2026 2:53:35 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786416707705494-12843-0/minicluster-data/ranger-kms/ews/webapp] .....
Aug 11, 2026 2:53:35 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786416707705494-12843-0/minicluster-data/ranger-kms/ews/webapp].
Aug 11, 2026 2:53:35 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
I20260811 02:53:36.812845 12843 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 6.032s user 0.027s sys 0.248s
I20260811 02:53:36.812988 12843 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260811 02:53:36.813093 12843 mini_ranger_kms.cc:401] 127.12.138.212:54539/kms/v1/keys
Aug 11 02:53:36 dist-test-slave-9gcw krb5kdc[25478](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786416816, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.12.138.212@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Aug 11 02:53:37 dist-test-slave-9gcw krb5kdc[25478](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416816, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.12.138.212@KRBTEST.COM for HTTP/127.12.138.212@KRBTEST.COM
Aug 11 02:53:37 dist-test-slave-9gcw krb5kdc[25478](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416816, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.12.138.212@KRBTEST.COM for HTTP/127.12.138.212@KRBTEST.COM
Aug 11 02:53:40 dist-test-slave-9gcw krb5kdc[25478](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416820, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.12.138.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.138.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.138.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786416707705494-12843-0/krb5kdc/kudu_127.12.138.254.keytab.
Entry for principal kudu/127.12.138.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786416707705494-12843-0/krb5kdc/kudu_127.12.138.254.keytab.
I20260811 02:53:40.301132 12843 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDawEwS/build/debug/bin/kudu
/tmp/dist-test-taskDawEwS/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786416707705494-12843-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786416707705494-12843-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786416707705494-12843-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786416707705494-12843-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.138.254:33515
--webserver_interface=127.12.138.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786416707705494-12843-0/krb5kdc/kudu_127.12.138.254.keytab
--principal=kudu/127.12.138.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.138.212:46169
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.12.138.254:33515
--ranger_config_path=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786416707705494-12843-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.12.138.212:54539/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786416707705494-12843-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786416707705494-12843-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786416707705494-12843-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 02:53:40.423907 26012 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 02:53:40.424185 26012 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 02:53:40.424226 26012 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 02:53:40.429344 26012 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 02:53:40.429426 26012 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 02:53:40.429448 26012 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 02:53:40.429467 26012 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 02:53:40.429486 26012 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 02:53:40.435837 26012 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.138.212:46169
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786416707705494-12843-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786416707705494-12843-0/minicluster-data/master-0/wal
--ranger_kms_url=127.12.138.212:54539/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.12.138.254:33515
--ranger_config_path=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786416707705494-12843-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786416707705494-12843-0/krb5kdc/kudu_127.12.138.254.keytab
--principal=kudu/127.12.138.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.138.254:33515
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786416707705494-12843-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.12.138.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26012
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786416707705494-12843-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 02:09:49 UTC on ccycloud.quasar-fsnqqc.root.comops.site
build id 4579570
I20260811 02:53:40.437029 26012 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 02:53:40.438154 26012 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 02:53:40.448256 26020 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:53:40.448273 26021 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:53:40.448601 26012 server_base.cc:1034] running on GCE node
W20260811 02:53:40.448920 26024 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:53:40.449572 26012 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 02:53:40.450817 26012 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 02:53:40.452029 26012 hybrid_clock.cc:648] HybridClock initialized: now 1786416820452008 us; error 41 us; skew 500 ppm
Aug 11 02:53:40 dist-test-slave-9gcw krb5kdc[25478](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416820, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.138.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 02:53:40.455503 26012 init.cc:377] Logged in from keytab as kudu/127.12.138.254@KRBTEST.COM (short username kudu)
I20260811 02:53:40.456820 26012 webserver.cc:460] Webserver started at http://127.12.138.254:37003/ using document root <none> and password file <none>
I20260811 02:53:40.457567 26012 fs_manager.cc:359] Metadata directory not provided
I20260811 02:53:40.457662 26012 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 02:53:40.457907 26012 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 11 02:53:40 dist-test-slave-9gcw krb5kdc[25478](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416820, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.138.254@KRBTEST.COM for HTTP/127.12.138.212@KRBTEST.COM
I20260811 02:53:40.506824 26012 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786416707705494-12843-0/minicluster-data/master-0/data/instance:
uuid: "b58647aa611d497f9fd438e24ca8baee"
format_stamp: "Formatted at 2026-08-11 02:53:40 on dist-test-slave-9gcw"
server_key: "accbaa39ca20d881f567023bedcb510b"
server_key_iv: "35219a6141d594fa11d2737dec0e75f5"
server_key_version: "kuduclusterkey@0"
I20260811 02:53:40.507575 26012 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786416707705494-12843-0/minicluster-data/master-0/wal/instance:
uuid: "b58647aa611d497f9fd438e24ca8baee"
format_stamp: "Formatted at 2026-08-11 02:53:40 on dist-test-slave-9gcw"
server_key: "accbaa39ca20d881f567023bedcb510b"
server_key_iv: "35219a6141d594fa11d2737dec0e75f5"
server_key_version: "kuduclusterkey@0"
I20260811 02:53:40.512306 26012 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260811 02:53:40.531754 26033 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 02:53:40.533289 26012 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 02:53:40.533434 26012 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786416707705494-12843-0/minicluster-data/master-0/data,/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786416707705494-12843-0/minicluster-data/master-0/wal
uuid: "b58647aa611d497f9fd438e24ca8baee"
format_stamp: "Formatted at 2026-08-11 02:53:40 on dist-test-slave-9gcw"
server_key: "accbaa39ca20d881f567023bedcb510b"
server_key_iv: "35219a6141d594fa11d2737dec0e75f5"
server_key_version: "kuduclusterkey@0"
I20260811 02:53:40.533574 26012 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786416707705494-12843-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786416707705494-12843-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786416707705494-12843-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 02:53:40.562835 26012 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 02:53:40.566057 26012 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 02:53:40.566265 26012 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 02:53:40.575016 26012 rpc_server.cc:307] RPC server started. Bound to: 127.12.138.254:33515
I20260811 02:53:40.575095 26091 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.138.254:33515 every 8 connection(s)
I20260811 02:53:40.576231 26012 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786416707705494-12843-0/minicluster-data/master-0/data/info.pb
I20260811 02:53:40.579864 26092 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 02:53:40.580824 12843 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDawEwS/build/debug/bin/kudu as pid 26012
I20260811 02:53:40.580961 12843 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786416707705494-12843-0/minicluster-data/master-0/wal/instance
I20260811 02:53:40.581339 12843 external_mini_cluster.cc:1428] Setting key 86e18013e00af2abdf4d2811c7e17b21
I20260811 02:53:40.588441 26092 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b58647aa611d497f9fd438e24ca8baee: Bootstrap starting.
Aug 11 02:53:40 dist-test-slave-9gcw krb5kdc[25478](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416820, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.12.138.254@KRBTEST.COM
I20260811 02:53:40.591425 26092 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b58647aa611d497f9fd438e24ca8baee: Neither blocks nor log segments found. Creating new log.
I20260811 02:53:40.592862 26092 log.cc:824] T 00000000000000000000000000000000 P b58647aa611d497f9fd438e24ca8baee: Log is configured to *not* fsync() on all Append() calls
I20260811 02:53:40.595317 26092 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b58647aa611d497f9fd438e24ca8baee: No bootstrap required, opened a new log
I20260811 02:53:40.596689 26095 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:53:40.582862 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44576 (local address 127.12.138.254:33515)
0811 02:53:40.583284 (+ 422us) server_negotiation.cc:206] Beginning negotiation
0811 02:53:40.583301 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0811 02:53:40.583337 (+ 36us) server_negotiation.cc:407] Connection header received
0811 02:53:40.584073 (+ 736us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:53:40.584091 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:53:40.584430 (+ 339us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:53:40.584821 (+ 391us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:53:40.585728 (+ 907us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:53:40.586788 (+ 1060us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:53:40.587674 (+ 886us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:53:40.588159 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:53:40.588327 (+ 168us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:53:40.591125 (+ 2798us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:53:40.591131 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:53:40.591145 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:53:40.591172 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:53:40.592995 (+ 1823us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:53:40.593612 (+ 617us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:53:40.593619 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:53:40.593626 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:53:40.593851 (+ 225us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:53:40.594220 (+ 369us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:53:40.594223 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:53:40.594225 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:53:40.594806 (+ 581us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:53:40.595013 (+ 207us) server_negotiation.cc:1035] Waiting for connection context
0811 02:53:40.595378 (+ 365us) server_negotiation.cc:299] Negotiation successful
0811 02:53:40.595698 (+ 320us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":263,"thread_start_us":143,"threads_started":1}
I20260811 02:53:40.599323 26092 raft_consensus.cc:348] T 00000000000000000000000000000000 P b58647aa611d497f9fd438e24ca8baee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b58647aa611d497f9fd438e24ca8baee" member_type: VOTER last_known_addr { host: "127.12.138.254" port: 33515 } }
I20260811 02:53:40.599627 26092 raft_consensus.cc:374] T 00000000000000000000000000000000 P b58647aa611d497f9fd438e24ca8baee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 02:53:40.599689 26092 raft_consensus.cc:729] T 00000000000000000000000000000000 P b58647aa611d497f9fd438e24ca8baee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b58647aa611d497f9fd438e24ca8baee, State: Initialized, Role: FOLLOWER
I20260811 02:53:40.600108 26092 consensus_queue.cc:260] T 00000000000000000000000000000000 P b58647aa611d497f9fd438e24ca8baee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b58647aa611d497f9fd438e24ca8baee" member_type: VOTER last_known_addr { host: "127.12.138.254" port: 33515 } }
I20260811 02:53:40.600262 26092 raft_consensus.cc:388] T 00000000000000000000000000000000 P b58647aa611d497f9fd438e24ca8baee [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 02:53:40.600332 26092 raft_consensus.cc:482] T 00000000000000000000000000000000 P b58647aa611d497f9fd438e24ca8baee [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 02:53:40.600432 26092 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b58647aa611d497f9fd438e24ca8baee [term 0 FOLLOWER]: Advancing to term 1
I20260811 02:53:40.601495 26092 raft_consensus.cc:504] T 00000000000000000000000000000000 P b58647aa611d497f9fd438e24ca8baee [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b58647aa611d497f9fd438e24ca8baee" member_type: VOTER last_known_addr { host: "127.12.138.254" port: 33515 } }
I20260811 02:53:40.601852 26092 leader_election.cc:302] T 00000000000000000000000000000000 P b58647aa611d497f9fd438e24ca8baee [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b58647aa611d497f9fd438e24ca8baee; no voters:
I20260811 02:53:40.602190 26092 leader_election.cc:288] T 00000000000000000000000000000000 P b58647aa611d497f9fd438e24ca8baee [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 02:53:40.602344 26098 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b58647aa611d497f9fd438e24ca8baee [term 1 FOLLOWER]: Leader election won for term 1
I20260811 02:53:40.602666 26098 raft_consensus.cc:686] T 00000000000000000000000000000000 P b58647aa611d497f9fd438e24ca8baee [term 1 LEADER]: Becoming Leader. State: Replica: b58647aa611d497f9fd438e24ca8baee, State: Running, Role: LEADER
I20260811 02:53:40.603073 26098 consensus_queue.cc:237] T 00000000000000000000000000000000 P b58647aa611d497f9fd438e24ca8baee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b58647aa611d497f9fd438e24ca8baee" member_type: VOTER last_known_addr { host: "127.12.138.254" port: 33515 } }
I20260811 02:53:40.603615 26092 sys_catalog.cc:564] T 00000000000000000000000000000000 P b58647aa611d497f9fd438e24ca8baee [sys.catalog]: configured and running, proceeding with master startup.
I20260811 02:53:40.604370 26098 sys_catalog.cc:455] T 00000000000000000000000000000000 P b58647aa611d497f9fd438e24ca8baee [sys.catalog]: SysCatalogTable state changed. Reason: New leader b58647aa611d497f9fd438e24ca8baee. Latest consensus state: current_term: 1 leader_uuid: "b58647aa611d497f9fd438e24ca8baee" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b58647aa611d497f9fd438e24ca8baee" member_type: VOTER last_known_addr { host: "127.12.138.254" port: 33515 } } }
I20260811 02:53:40.604496 26098 sys_catalog.cc:458] T 00000000000000000000000000000000 P b58647aa611d497f9fd438e24ca8baee [sys.catalog]: This master's current role is: LEADER
I20260811 02:53:40.604925 26106 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 02:53:40.606163 26099 sys_catalog.cc:455] T 00000000000000000000000000000000 P b58647aa611d497f9fd438e24ca8baee [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b58647aa611d497f9fd438e24ca8baee" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b58647aa611d497f9fd438e24ca8baee" member_type: VOTER last_known_addr { host: "127.12.138.254" port: 33515 } } }
I20260811 02:53:40.606267 26099 sys_catalog.cc:458] T 00000000000000000000000000000000 P b58647aa611d497f9fd438e24ca8baee [sys.catalog]: This master's current role is: LEADER
I20260811 02:53:40.608433 26106 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 02:53:40.608858 26092 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786416707705494-12843-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260811 02:53:40.615573 26106 catalog_manager.cc:1347] Generated new cluster ID: dbd9dbebd6a446bc9aa4fef05596c147
I20260811 02:53:40.615724 26106 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 02:53:40.632556 26106 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 02:53:40.634195 26106 catalog_manager.cc:1504] Loading token signing keys...
I20260811 02:53:40.640669 26106 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b58647aa611d497f9fd438e24ca8baee: Generated new TSK 0
I20260811 02:53:40.641745 26106 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.12.138.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.138.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.138.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786416707705494-12843-0/krb5kdc/kudu_127.12.138.193.keytab.
Entry for principal kudu/127.12.138.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786416707705494-12843-0/krb5kdc/kudu_127.12.138.193.keytab.
I20260811 02:53:41.444906 12843 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDawEwS/build/debug/bin/kudu
/tmp/dist-test-taskDawEwS/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786416707705494-12843-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786416707705494-12843-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786416707705494-12843-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786416707705494-12843-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.12.138.212:54539/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.138.193:0
--local_ip_for_outbound_sockets=127.12.138.193
--webserver_interface=127.12.138.193
--webserver_port=0
--tserver_master_addrs=127.12.138.254:33515
--keytab_file=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786416707705494-12843-0/krb5kdc/kudu_127.12.138.193.keytab
--principal=kudu/127.12.138.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.138.212:46169
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786416707705494-12843-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786416707705494-12843-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786416707705494-12843-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 02:53:41.572229 26145 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 02:53:41.572566 26145 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 02:53:41.572650 26145 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 02:53:41.577854 26145 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 02:53:41.577984 26145 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 02:53:41.578099 26145 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.138.193
I20260811 02:53:41.585695 26145 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.138.212:46169
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786416707705494-12843-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786416707705494-12843-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.12.138.212:54539/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786416707705494-12843-0/krb5kdc/kudu_127.12.138.193.keytab
--principal=kudu/127.12.138.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.138.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786416707705494-12843-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.12.138.193
--webserver_port=0
--tserver_master_addrs=127.12.138.254:33515
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26145
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.138.193
--log_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786416707705494-12843-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 02:09:49 UTC on ccycloud.quasar-fsnqqc.root.comops.site
build id 4579570
I20260811 02:53:41.587188 26145 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 02:53:41.588308 26145 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 02:53:41.599520 26152 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:53:41.599520 26153 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:53:41.600044 26156 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:53:41.600456 26145 server_base.cc:1034] running on GCE node
I20260811 02:53:41.600919 26145 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 02:53:41.601622 26145 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 02:53:41.602819 26145 hybrid_clock.cc:648] HybridClock initialized: now 1786416821602781 us; error 51 us; skew 500 ppm
Aug 11 02:53:41 dist-test-slave-9gcw krb5kdc[25478](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416821, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.138.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 02:53:41.606102 26145 init.cc:377] Logged in from keytab as kudu/127.12.138.193@KRBTEST.COM (short username kudu)
I20260811 02:53:41.607057 26145 webserver.cc:460] Webserver started at http://127.12.138.193:38225/ using document root <none> and password file <none>
I20260811 02:53:41.607690 26145 fs_manager.cc:359] Metadata directory not provided
I20260811 02:53:41.607759 26145 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 02:53:41.607980 26145 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 11 02:53:41 dist-test-slave-9gcw krb5kdc[25478](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416821, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.138.193@KRBTEST.COM for HTTP/127.12.138.212@KRBTEST.COM
I20260811 02:53:41.624593 26145 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786416707705494-12843-0/minicluster-data/ts-0/data/instance:
uuid: "c257dbb63ff84888891d41f05882f5e1"
format_stamp: "Formatted at 2026-08-11 02:53:41 on dist-test-slave-9gcw"
server_key: "ac2eb2e73a273b10afcd30dfb21afd2e"
server_key_iv: "209ac73da278b17c175493d6f6cdd38e"
server_key_version: "kuduclusterkey@0"
I20260811 02:53:41.625526 26145 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786416707705494-12843-0/minicluster-data/ts-0/wal/instance:
uuid: "c257dbb63ff84888891d41f05882f5e1"
format_stamp: "Formatted at 2026-08-11 02:53:41 on dist-test-slave-9gcw"
server_key: "ac2eb2e73a273b10afcd30dfb21afd2e"
server_key_iv: "209ac73da278b17c175493d6f6cdd38e"
server_key_version: "kuduclusterkey@0"
I20260811 02:53:41.630206 26145 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.001s
I20260811 02:53:41.650882 26166 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 02:53:41.652356 26145 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260811 02:53:41.652529 26145 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786416707705494-12843-0/minicluster-data/ts-0/data,/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786416707705494-12843-0/minicluster-data/ts-0/wal
uuid: "c257dbb63ff84888891d41f05882f5e1"
format_stamp: "Formatted at 2026-08-11 02:53:41 on dist-test-slave-9gcw"
server_key: "ac2eb2e73a273b10afcd30dfb21afd2e"
server_key_iv: "209ac73da278b17c175493d6f6cdd38e"
server_key_version: "kuduclusterkey@0"
I20260811 02:53:41.652763 26145 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786416707705494-12843-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786416707705494-12843-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786416707705494-12843-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 02:53:41.674700 26145 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 02:53:41.678903 26145 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 02:53:41.679226 26145 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 02:53:41.680080 26145 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 02:53:41.681253 26145 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 02:53:41.681367 26145 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 02:53:41.681494 26145 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 02:53:41.681571 26145 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 02:53:41.693372 26145 rpc_server.cc:307] RPC server started. Bound to: 127.12.138.193:46843
I20260811 02:53:41.694813 26145 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786416707705494-12843-0/minicluster-data/ts-0/data/info.pb
I20260811 02:53:41.695901 12843 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDawEwS/build/debug/bin/kudu as pid 26145
I20260811 02:53:41.696053 12843 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786416707705494-12843-0/minicluster-data/ts-0/wal/instance
I20260811 02:53:41.696370 12843 external_mini_cluster.cc:1428] Setting key 860498cd100d113a85e71af59830d704
I20260811 02:53:41.705740 26306 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.138.193:46843 every 8 connection(s)
WARNING: no policy specified for kudu/127.12.138.194@KRBTEST.COM; defaulting to no policy
Aug 11 02:53:41 dist-test-slave-9gcw krb5kdc[25478](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416821, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.138.193@KRBTEST.COM for kudu/127.12.138.254@KRBTEST.COM
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.138.194@KRBTEST.COM" created.
I20260811 02:53:41.748068 26311 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:53:41.717501 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.138.193:58611 (local address 127.12.138.254:33515)
0811 02:53:41.718708 (+ 1207us) server_negotiation.cc:206] Beginning negotiation
0811 02:53:41.718713 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 02:53:41.718747 (+ 34us) server_negotiation.cc:407] Connection header received
0811 02:53:41.722071 (+ 3324us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:53:41.722076 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:53:41.722156 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:53:41.722276 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:53:41.724008 (+ 1732us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:53:41.724735 (+ 727us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:53:41.726853 (+ 2118us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:53:41.727271 (+ 418us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:53:41.727382 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:53:41.743079 (+ 15697us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:53:41.743085 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:53:41.743088 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:53:41.743124 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:53:41.745310 (+ 2186us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:53:41.746085 (+ 775us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:53:41.746089 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:53:41.746091 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:53:41.746188 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:53:41.746631 (+ 443us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:53:41.746634 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:53:41.746636 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:53:41.746880 (+ 244us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:53:41.746978 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0811 02:53:41.747719 (+ 741us) server_negotiation.cc:299] Negotiation successful
0811 02:53:41.747868 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1120,"thread_start_us":106,"threads_started":1}
I20260811 02:53:41.750092 26310 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:53:41.716180 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.138.254:33515 (local address 127.12.138.193:58611)
0811 02:53:41.718002 (+ 1822us) negotiation.cc:107] Waiting for socket to connect
0811 02:53:41.718071 (+ 69us) client_negotiation.cc:174] Beginning negotiation
0811 02:53:41.719274 (+ 1203us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:53:41.722575 (+ 3301us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:53:41.722587 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:53:41.723038 (+ 451us) client_negotiation.cc:189] Negotiated authn=SASL
0811 02:53:41.723818 (+ 780us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:53:41.723836 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:53:41.724917 (+ 1081us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:53:41.724922 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:53:41.726681 (+ 1759us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:53:41.726697 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:53:41.727589 (+ 892us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:53:41.727593 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:53:41.727812 (+ 219us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:53:41.728631 (+ 819us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 02:53:41.728661 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0811 02:53:41.742794 (+ 14133us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 02:53:41.745482 (+ 2688us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:53:41.745490 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:53:41.745512 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:53:41.745954 (+ 442us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:53:41.746312 (+ 358us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:53:41.746316 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:53:41.746318 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:53:41.746521 (+ 203us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:53:41.747037 (+ 516us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 02:53:41.747044 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 02:53:41.747425 (+ 381us) client_negotiation.cc:769] Sending connection context
0811 02:53:41.748589 (+ 1164us) client_negotiation.cc:240] Negotiation successful
0811 02:53:41.748960 (+ 371us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1553,"thread_start_us":176,"threads_started":1}
I20260811 02:53:41.754487 26307 heartbeater.cc:347] Connected to a master server at 127.12.138.254:33515
I20260811 02:53:41.754753 26307 heartbeater.cc:464] Registering TS with master...
I20260811 02:53:41.755470 26307 heartbeater.cc:511] Master 127.12.138.254:33515 requested a full tablet report, sending...
I20260811 02:53:41.757552 26052 ts_manager.cc:194] Registered new tserver with Master: c257dbb63ff84888891d41f05882f5e1 (127.12.138.193:46843)
I20260811 02:53:41.759135 26052 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.12.138.193@KRBTEST.COM'} at 127.12.138.193:58611
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.138.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786416707705494-12843-0/krb5kdc/kudu_127.12.138.194.keytab.
Entry for principal kudu/127.12.138.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786416707705494-12843-0/krb5kdc/kudu_127.12.138.194.keytab.
I20260811 02:53:41.782759 12843 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDawEwS/build/debug/bin/kudu
/tmp/dist-test-taskDawEwS/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786416707705494-12843-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786416707705494-12843-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786416707705494-12843-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786416707705494-12843-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.12.138.212:54539/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.138.194:0
--local_ip_for_outbound_sockets=127.12.138.194
--webserver_interface=127.12.138.194
--webserver_port=0
--tserver_master_addrs=127.12.138.254:33515
--keytab_file=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786416707705494-12843-0/krb5kdc/kudu_127.12.138.194.keytab
--principal=kudu/127.12.138.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.138.212:46169
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786416707705494-12843-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786416707705494-12843-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786416707705494-12843-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 11 02:53:41 dist-test-slave-9gcw krb5kdc[25478](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786416821, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.138.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260811 02:53:41.948302 26315 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 02:53:41.948689 26315 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 02:53:41.948762 26315 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 02:53:41.955025 26315 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 02:53:41.955114 26315 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 02:53:41.955215 26315 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.138.194
I20260811 02:53:41.963493 26315 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.138.212:46169
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786416707705494-12843-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786416707705494-12843-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.12.138.212:54539/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786416707705494-12843-0/krb5kdc/kudu_127.12.138.194.keytab
--principal=kudu/127.12.138.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.138.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786416707705494-12843-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.12.138.194
--webserver_port=0
--tserver_master_addrs=127.12.138.254:33515
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26315
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.138.194
--log_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786416707705494-12843-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 02:09:49 UTC on ccycloud.quasar-fsnqqc.root.comops.site
build id 4579570
I20260811 02:53:41.965076 26315 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 02:53:41.966454 26315 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 02:53:41.979622 26325 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:53:41.979686 26324 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:53:41.979918 26327 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:53:41.981408 26315 server_base.cc:1034] running on GCE node
I20260811 02:53:41.981896 26315 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 02:53:41.982664 26315 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 02:53:41.983844 26315 hybrid_clock.cc:648] HybridClock initialized: now 1786416821983766 us; error 92 us; skew 500 ppm
Aug 11 02:53:41 dist-test-slave-9gcw krb5kdc[25478](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416821, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.138.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 02:53:41.990178 26315 init.cc:377] Logged in from keytab as kudu/127.12.138.194@KRBTEST.COM (short username kudu)
I20260811 02:53:41.991549 26315 webserver.cc:460] Webserver started at http://127.12.138.194:43069/ using document root <none> and password file <none>
I20260811 02:53:41.992434 26315 fs_manager.cc:359] Metadata directory not provided
I20260811 02:53:41.992630 26315 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 02:53:41.992982 26315 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 11 02:53:41 dist-test-slave-9gcw krb5kdc[25478](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416821, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.138.194@KRBTEST.COM for HTTP/127.12.138.212@KRBTEST.COM
I20260811 02:53:42.019726 26315 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786416707705494-12843-0/minicluster-data/ts-1/data/instance:
uuid: "fae440a2b7cd48928353d69dc66c925e"
format_stamp: "Formatted at 2026-08-11 02:53:42 on dist-test-slave-9gcw"
server_key: "2a1c7778ab39a24a5dd7595c9c0fd08a"
server_key_iv: "9b689e7c4b6af118c628534518ea8402"
server_key_version: "kuduclusterkey@0"
I20260811 02:53:42.020684 26315 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786416707705494-12843-0/minicluster-data/ts-1/wal/instance:
uuid: "fae440a2b7cd48928353d69dc66c925e"
format_stamp: "Formatted at 2026-08-11 02:53:42 on dist-test-slave-9gcw"
server_key: "2a1c7778ab39a24a5dd7595c9c0fd08a"
server_key_iv: "9b689e7c4b6af118c628534518ea8402"
server_key_version: "kuduclusterkey@0"
I20260811 02:53:42.030853 26315 fs_manager.cc:691] Time spent creating directory manager: real 0.010s user 0.003s sys 0.001s
I20260811 02:53:42.050917 26340 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 02:53:42.052894 26315 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.004s sys 0.000s
I20260811 02:53:42.053170 26315 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786416707705494-12843-0/minicluster-data/ts-1/data,/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786416707705494-12843-0/minicluster-data/ts-1/wal
uuid: "fae440a2b7cd48928353d69dc66c925e"
format_stamp: "Formatted at 2026-08-11 02:53:42 on dist-test-slave-9gcw"
server_key: "2a1c7778ab39a24a5dd7595c9c0fd08a"
server_key_iv: "9b689e7c4b6af118c628534518ea8402"
server_key_version: "kuduclusterkey@0"
I20260811 02:53:42.053418 26315 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786416707705494-12843-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786416707705494-12843-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786416707705494-12843-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 02:53:42.078425 26315 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 02:53:42.082712 26315 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 02:53:42.083010 26315 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 02:53:42.083886 26315 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 02:53:42.085130 26315 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 02:53:42.085296 26315 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.001s
I20260811 02:53:42.085400 26315 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 02:53:42.085469 26315 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 02:53:42.100731 26315 rpc_server.cc:307] RPC server started. Bound to: 127.12.138.194:33951
I20260811 02:53:42.102279 26315 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786416707705494-12843-0/minicluster-data/ts-1/data/info.pb
I20260811 02:53:42.104267 26476 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.138.194:33951 every 8 connection(s)
I20260811 02:53:42.106041 12843 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDawEwS/build/debug/bin/kudu as pid 26315
I20260811 02:53:42.106185 12843 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786416707705494-12843-0/minicluster-data/ts-1/wal/instance
I20260811 02:53:42.106531 12843 external_mini_cluster.cc:1428] Setting key 00365d528113886077fd7376b625faa0
WARNING: no policy specified for kudu/127.12.138.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.138.195@KRBTEST.COM" created.
Aug 11 02:53:42 dist-test-slave-9gcw krb5kdc[25478](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416821, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.138.194@KRBTEST.COM for kudu/127.12.138.254@KRBTEST.COM
I20260811 02:53:42.135985 26311 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:53:42.107131 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.138.194:39065 (local address 127.12.138.254:33515)
0811 02:53:42.107288 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0811 02:53:42.107293 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 02:53:42.117895 (+ 10602us) server_negotiation.cc:407] Connection header received
0811 02:53:42.123202 (+ 5307us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:53:42.123208 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:53:42.123283 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:53:42.123345 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:53:42.125208 (+ 1863us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:53:42.125918 (+ 710us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:53:42.127231 (+ 1313us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:53:42.127574 (+ 343us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:53:42.127710 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:53:42.131283 (+ 3573us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:53:42.131287 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:53:42.131289 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:53:42.131334 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:53:42.133269 (+ 1935us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:53:42.133972 (+ 703us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:53:42.133978 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:53:42.133983 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:53:42.134091 (+ 108us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:53:42.134563 (+ 472us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:53:42.134569 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:53:42.134572 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:53:42.134864 (+ 292us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:53:42.134944 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0811 02:53:42.135641 (+ 697us) server_negotiation.cc:299] Negotiation successful
0811 02:53:42.135801 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260811 02:53:42.137027 26481 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:53:42.107587 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.138.254:33515 (local address 127.12.138.194:39065)
0811 02:53:42.117649 (+ 10062us) negotiation.cc:107] Waiting for socket to connect
0811 02:53:42.117740 (+ 91us) client_negotiation.cc:174] Beginning negotiation
0811 02:53:42.122868 (+ 5128us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:53:42.123893 (+ 1025us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:53:42.123904 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:53:42.124296 (+ 392us) client_negotiation.cc:189] Negotiated authn=SASL
0811 02:53:42.125002 (+ 706us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:53:42.125018 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:53:42.126060 (+ 1042us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:53:42.126064 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:53:42.127099 (+ 1035us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:53:42.127109 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:53:42.127969 (+ 860us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:53:42.127976 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:53:42.128208 (+ 232us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:53:42.128984 (+ 776us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 02:53:42.129011 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0811 02:53:42.131037 (+ 2026us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 02:53:42.133414 (+ 2377us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:53:42.133423 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:53:42.133444 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:53:42.133832 (+ 388us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:53:42.134224 (+ 392us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:53:42.134229 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:53:42.134234 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:53:42.134434 (+ 200us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:53:42.135036 (+ 602us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 02:53:42.135044 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 02:53:42.135388 (+ 344us) client_negotiation.cc:769] Sending connection context
0811 02:53:42.135602 (+ 214us) client_negotiation.cc:240] Negotiation successful
0811 02:53:42.135944 (+ 342us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":9697,"thread_start_us":146,"threads_started":1}
I20260811 02:53:42.138684 26478 heartbeater.cc:347] Connected to a master server at 127.12.138.254:33515
I20260811 02:53:42.138948 26478 heartbeater.cc:464] Registering TS with master...
I20260811 02:53:42.139536 26478 heartbeater.cc:511] Master 127.12.138.254:33515 requested a full tablet report, sending...
I20260811 02:53:42.140779 26052 ts_manager.cc:194] Registered new tserver with Master: fae440a2b7cd48928353d69dc66c925e (127.12.138.194:33951)
I20260811 02:53:42.141544 26052 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.12.138.194@KRBTEST.COM'} at 127.12.138.194:39065
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.138.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786416707705494-12843-0/krb5kdc/kudu_127.12.138.195.keytab.
Entry for principal kudu/127.12.138.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786416707705494-12843-0/krb5kdc/kudu_127.12.138.195.keytab.
I20260811 02:53:42.152772 12843 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDawEwS/build/debug/bin/kudu
/tmp/dist-test-taskDawEwS/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786416707705494-12843-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786416707705494-12843-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786416707705494-12843-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786416707705494-12843-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.12.138.212:54539/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.138.195:0
--local_ip_for_outbound_sockets=127.12.138.195
--webserver_interface=127.12.138.195
--webserver_port=0
--tserver_master_addrs=127.12.138.254:33515
--keytab_file=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786416707705494-12843-0/krb5kdc/kudu_127.12.138.195.keytab
--principal=kudu/127.12.138.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.138.212:46169
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786416707705494-12843-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786416707705494-12843-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786416707705494-12843-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 02:53:42.335106 26484 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 02:53:42.335587 26484 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 02:53:42.335886 26484 flags.cc:425] Enabled unsafe flag: --never_fsync=true
Aug 11 02:53:42 dist-test-slave-9gcw krb5kdc[25478](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416821, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.138.254@KRBTEST.COM for HTTP/127.12.138.212@KRBTEST.COM
W20260811 02:53:42.342442 26484 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 02:53:42.342566 26484 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 02:53:42.342727 26484 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.138.195
I20260811 02:53:42.350732 26484 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.138.212:46169
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786416707705494-12843-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786416707705494-12843-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.12.138.212:54539/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786416707705494-12843-0/krb5kdc/kudu_127.12.138.195.keytab
--principal=kudu/127.12.138.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.138.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786416707705494-12843-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.12.138.195
--webserver_port=0
--tserver_master_addrs=127.12.138.254:33515
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26484
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.138.195
--log_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786416707705494-12843-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 02:09:49 UTC on ccycloud.quasar-fsnqqc.root.comops.site
build id 4579570
I20260811 02:53:42.352330 26484 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 02:53:42.353725 26484 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 02:53:42.379268 26493 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:53:42.379308 26495 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:53:42.379843 26484 server_base.cc:1034] running on GCE node
W20260811 02:53:42.380170 26498 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:53:42.380672 26484 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 02:53:42.381616 26484 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 02:53:42.382902 26484 hybrid_clock.cc:648] HybridClock initialized: now 1786416822382876 us; error 43 us; skew 500 ppm
Aug 11 02:53:42 dist-test-slave-9gcw krb5kdc[25478](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416822, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.138.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 02:53:42.386837 26484 init.cc:377] Logged in from keytab as kudu/127.12.138.195@KRBTEST.COM (short username kudu)
I20260811 02:53:42.387964 26484 webserver.cc:460] Webserver started at http://127.12.138.195:44017/ using document root <none> and password file <none>
I20260811 02:53:42.388587 26484 fs_manager.cc:359] Metadata directory not provided
I20260811 02:53:42.388690 26484 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 02:53:42.388916 26484 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 11 02:53:42 dist-test-slave-9gcw krb5kdc[25478](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416822, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.138.195@KRBTEST.COM for HTTP/127.12.138.212@KRBTEST.COM
I20260811 02:53:42.409946 26484 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786416707705494-12843-0/minicluster-data/ts-2/data/instance:
uuid: "64c6f3fa2b7d4d7a80158a86e4e8e736"
format_stamp: "Formatted at 2026-08-11 02:53:42 on dist-test-slave-9gcw"
server_key: "7106efe36f28499772f404028deb44dc"
server_key_iv: "9fd608419187ca61b1e15757d70e75ab"
server_key_version: "kuduclusterkey@0"
I20260811 02:53:42.412695 26484 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786416707705494-12843-0/minicluster-data/ts-2/wal/instance:
uuid: "64c6f3fa2b7d4d7a80158a86e4e8e736"
format_stamp: "Formatted at 2026-08-11 02:53:42 on dist-test-slave-9gcw"
server_key: "7106efe36f28499772f404028deb44dc"
server_key_iv: "9fd608419187ca61b1e15757d70e75ab"
server_key_version: "kuduclusterkey@0"
I20260811 02:53:42.425412 26484 fs_manager.cc:691] Time spent creating directory manager: real 0.012s user 0.002s sys 0.003s
I20260811 02:53:42.450151 26507 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 02:53:42.451856 26484 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260811 02:53:42.451985 26484 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786416707705494-12843-0/minicluster-data/ts-2/data,/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786416707705494-12843-0/minicluster-data/ts-2/wal
uuid: "64c6f3fa2b7d4d7a80158a86e4e8e736"
format_stamp: "Formatted at 2026-08-11 02:53:42 on dist-test-slave-9gcw"
server_key: "7106efe36f28499772f404028deb44dc"
server_key_iv: "9fd608419187ca61b1e15757d70e75ab"
server_key_version: "kuduclusterkey@0"
I20260811 02:53:42.452132 26484 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786416707705494-12843-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786416707705494-12843-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786416707705494-12843-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 02:53:42.490159 26484 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 02:53:42.495054 26484 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 02:53:42.495434 26484 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 02:53:42.496354 26484 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 02:53:42.497813 26484 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 02:53:42.497947 26484 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.001s sys 0.000s
I20260811 02:53:42.498077 26484 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 02:53:42.498173 26484 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 02:53:42.512897 26484 rpc_server.cc:307] RPC server started. Bound to: 127.12.138.195:34729
I20260811 02:53:42.514489 26484 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786416707705494-12843-0/minicluster-data/ts-2/data/info.pb
I20260811 02:53:42.521068 12843 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDawEwS/build/debug/bin/kudu as pid 26484
I20260811 02:53:42.521281 12843 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786416707705494-12843-0/minicluster-data/ts-2/wal/instance
I20260811 02:53:42.521924 12843 external_mini_cluster.cc:1428] Setting key 5b2cc5c9450263bd58de2e28a7c16ef6
I20260811 02:53:42.519658 26642 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.138.195:34729 every 8 connection(s)
Aug 11 02:53:42 dist-test-slave-9gcw krb5kdc[25478](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416822, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.138.195@KRBTEST.COM for kudu/127.12.138.254@KRBTEST.COM
I20260811 02:53:42.550098 26311 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:53:42.532727 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.138.195:44197 (local address 127.12.138.254:33515)
0811 02:53:42.532877 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0811 02:53:42.532882 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 02:53:42.533560 (+ 678us) server_negotiation.cc:407] Connection header received
0811 02:53:42.535112 (+ 1552us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:53:42.535117 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:53:42.535184 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:53:42.535253 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:53:42.537334 (+ 2081us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:53:42.538280 (+ 946us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:53:42.540322 (+ 2042us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:53:42.540708 (+ 386us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:53:42.540810 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:53:42.544698 (+ 3888us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:53:42.544702 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:53:42.544705 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:53:42.544739 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:53:42.546953 (+ 2214us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:53:42.547742 (+ 789us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:53:42.547749 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:53:42.547751 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:53:42.547857 (+ 106us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:53:42.548418 (+ 561us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:53:42.548422 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:53:42.548424 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:53:42.548715 (+ 291us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:53:42.548829 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0811 02:53:42.549720 (+ 891us) server_negotiation.cc:299] Negotiation successful
0811 02:53:42.549875 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260811 02:53:42.552342 26645 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:53:42.531393 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.138.254:33515 (local address 127.12.138.195:44197)
0811 02:53:42.533287 (+ 1894us) negotiation.cc:107] Waiting for socket to connect
0811 02:53:42.533378 (+ 91us) client_negotiation.cc:174] Beginning negotiation
0811 02:53:42.534783 (+ 1405us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:53:42.535775 (+ 992us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:53:42.535785 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:53:42.536251 (+ 466us) client_negotiation.cc:189] Negotiated authn=SASL
0811 02:53:42.537113 (+ 862us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:53:42.537158 (+ 45us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:53:42.538515 (+ 1357us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:53:42.538520 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:53:42.540139 (+ 1619us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:53:42.540153 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:53:42.541062 (+ 909us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:53:42.541067 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:53:42.541374 (+ 307us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:53:42.542227 (+ 853us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 02:53:42.542254 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0811 02:53:42.544453 (+ 2199us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 02:53:42.547154 (+ 2701us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:53:42.547161 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:53:42.547181 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:53:42.547586 (+ 405us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:53:42.548000 (+ 414us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:53:42.548004 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:53:42.548006 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:53:42.548272 (+ 266us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:53:42.548913 (+ 641us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 02:53:42.548922 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 02:53:42.549361 (+ 439us) client_negotiation.cc:769] Sending connection context
0811 02:53:42.550658 (+ 1297us) client_negotiation.cc:240] Negotiation successful
0811 02:53:42.551083 (+ 425us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1589,"thread_start_us":194,"threads_started":1}
I20260811 02:53:42.554301 26643 heartbeater.cc:347] Connected to a master server at 127.12.138.254:33515
I20260811 02:53:42.554601 26643 heartbeater.cc:464] Registering TS with master...
I20260811 02:53:42.555369 26643 heartbeater.cc:511] Master 127.12.138.254:33515 requested a full tablet report, sending...
I20260811 02:53:42.556854 26052 ts_manager.cc:194] Registered new tserver with Master: 64c6f3fa2b7d4d7a80158a86e4e8e736 (127.12.138.195:34729)
I20260811 02:53:42.557822 26052 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.12.138.195@KRBTEST.COM'} at 127.12.138.195:44197
I20260811 02:53:42.561385 12843 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 02:53:42.576872 26311 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:53:42.564268 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52512 (local address 127.12.138.254:33515)
0811 02:53:42.564414 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0811 02:53:42.564420 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 02:53:42.565324 (+ 904us) server_negotiation.cc:407] Connection header received
0811 02:53:42.565582 (+ 258us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:53:42.565602 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:53:42.565664 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:53:42.565731 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:53:42.567063 (+ 1332us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:53:42.567948 (+ 885us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:53:42.569459 (+ 1511us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:53:42.569865 (+ 406us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:53:42.569976 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:53:42.572353 (+ 2377us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:53:42.572358 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:53:42.572361 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:53:42.572394 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:53:42.574623 (+ 2229us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:53:42.575243 (+ 620us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:53:42.575248 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:53:42.575250 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:53:42.575369 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:53:42.575758 (+ 389us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:53:42.575762 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:53:42.575764 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:53:42.576004 (+ 240us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:53:42.576118 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0811 02:53:42.576536 (+ 418us) server_negotiation.cc:299] Negotiation successful
0811 02:53:42.576692 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260811 02:53:42.581295 26052 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52512:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 02:53:42.586853 26052 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 02:53:42.618992 26663 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:53:42.601738 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.138.195:34729 (local address 127.0.0.1:54144)
0811 02:53:42.607540 (+ 5802us) negotiation.cc:107] Waiting for socket to connect
0811 02:53:42.607677 (+ 137us) client_negotiation.cc:174] Beginning negotiation
0811 02:53:42.607839 (+ 162us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:53:42.610814 (+ 2975us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:53:42.610821 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:53:42.610849 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 02:53:42.611015 (+ 166us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:53:42.611025 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:53:42.613858 (+ 2833us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:53:42.613862 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:53:42.615615 (+ 1753us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:53:42.615629 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:53:42.618279 (+ 2650us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:53:42.618285 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:53:42.618419 (+ 134us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:53:42.618468 (+ 49us) client_negotiation.cc:769] Sending connection context
0811 02:53:42.618628 (+ 160us) client_negotiation.cc:240] Negotiation successful
0811 02:53:42.618783 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":5590,"thread_start_us":129,"threads_started":1}
I20260811 02:53:42.620083 26660 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:53:42.602948 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54144 (local address 127.12.138.195:34729)
0811 02:53:42.608548 (+ 5600us) server_negotiation.cc:206] Beginning negotiation
0811 02:53:42.608565 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0811 02:53:42.608609 (+ 44us) server_negotiation.cc:407] Connection header received
0811 02:53:42.608700 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:53:42.608706 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:53:42.608948 (+ 242us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:53:42.610569 (+ 1621us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 02:53:42.611206 (+ 637us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:53:42.613693 (+ 2487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:53:42.615817 (+ 2124us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:53:42.617867 (+ 2050us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:53:42.618018 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:53:42.618193 (+ 175us) server_negotiation.cc:1035] Waiting for connection context
0811 02:53:42.619758 (+ 1565us) server_negotiation.cc:299] Negotiation successful
0811 02:53:42.619902 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":5473,"thread_start_us":171,"threads_started":1}
I20260811 02:53:42.623368 26661 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:53:42.605738 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.138.193:46843 (local address 127.0.0.1:37740)
0811 02:53:42.610208 (+ 4470us) negotiation.cc:107] Waiting for socket to connect
0811 02:53:42.610225 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 02:53:42.610332 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:53:42.614155 (+ 3823us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:53:42.614158 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:53:42.614175 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 02:53:42.614268 (+ 93us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:53:42.614274 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:53:42.618336 (+ 4062us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:53:42.618340 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:53:42.619458 (+ 1118us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:53:42.619470 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:53:42.622977 (+ 3507us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:53:42.622981 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:53:42.623075 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:53:42.623103 (+ 28us) client_negotiation.cc:769] Sending connection context
0811 02:53:42.623154 (+ 51us) client_negotiation.cc:240] Negotiation successful
0811 02:53:42.623225 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4395,"thread_start_us":169,"threads_started":1}
I20260811 02:53:42.624295 26664 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:53:42.605883 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37740 (local address 127.12.138.193:46843)
0811 02:53:42.613357 (+ 7474us) server_negotiation.cc:206] Beginning negotiation
0811 02:53:42.613365 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 02:53:42.613393 (+ 28us) server_negotiation.cc:407] Connection header received
0811 02:53:42.613470 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:53:42.613476 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:53:42.613698 (+ 222us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:53:42.613997 (+ 299us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 02:53:42.615027 (+ 1030us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:53:42.616988 (+ 1961us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:53:42.619653 (+ 2665us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:53:42.620453 (+ 800us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:53:42.623865 (+ 3412us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:53:42.624003 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0811 02:53:42.624055 (+ 52us) server_negotiation.cc:299] Negotiation successful
0811 02:53:42.624150 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":7354,"thread_start_us":70,"threads_started":1}
I20260811 02:53:42.624097 26564 tablet_service.cc:1467] Processing CreateTablet for tablet bd43f8a07f6d4e98bcc29db965ecd4f8 (DEFAULT_TABLE table=test-table [id=2367bd72f7cd44d6a5727d7c3ef3dbc0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 02:53:42.626693 26564 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bd43f8a07f6d4e98bcc29db965ecd4f8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 02:53:42.628206 26659 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:53:42.604876 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.138.194:33951 (local address 127.0.0.1:39718)
0811 02:53:42.608131 (+ 3255us) negotiation.cc:107] Waiting for socket to connect
0811 02:53:42.608147 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 02:53:42.608245 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:53:42.615899 (+ 7654us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:53:42.615902 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:53:42.615918 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 02:53:42.615995 (+ 77us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:53:42.616000 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:53:42.620593 (+ 4593us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:53:42.620596 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:53:42.621801 (+ 1205us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:53:42.621813 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:53:42.627788 (+ 5975us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:53:42.627794 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:53:42.627903 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:53:42.627931 (+ 28us) client_negotiation.cc:769] Sending connection context
0811 02:53:42.627988 (+ 57us) client_negotiation.cc:240] Negotiation successful
0811 02:53:42.628065 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3179,"thread_start_us":109,"threads_started":1}
I20260811 02:53:42.630149 26662 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:53:42.605033 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39718 (local address 127.12.138.194:33951)
0811 02:53:42.614529 (+ 9496us) server_negotiation.cc:206] Beginning negotiation
0811 02:53:42.614546 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0811 02:53:42.614570 (+ 24us) server_negotiation.cc:407] Connection header received
0811 02:53:42.614756 (+ 186us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:53:42.614762 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:53:42.615539 (+ 777us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:53:42.615754 (+ 215us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 02:53:42.617116 (+ 1362us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:53:42.618196 (+ 1080us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:53:42.621980 (+ 3784us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:53:42.622825 (+ 845us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:53:42.629675 (+ 6850us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:53:42.629836 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0811 02:53:42.629901 (+ 65us) server_negotiation.cc:299] Negotiation successful
0811 02:53:42.630008 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":9386,"thread_start_us":106,"threads_started":1}
I20260811 02:53:42.632339 26222 tablet_service.cc:1467] Processing CreateTablet for tablet bd43f8a07f6d4e98bcc29db965ecd4f8 (DEFAULT_TABLE table=test-table [id=2367bd72f7cd44d6a5727d7c3ef3dbc0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 02:53:42.633309 26399 tablet_service.cc:1467] Processing CreateTablet for tablet bd43f8a07f6d4e98bcc29db965ecd4f8 (DEFAULT_TABLE table=test-table [id=2367bd72f7cd44d6a5727d7c3ef3dbc0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 02:53:42.634424 26399 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bd43f8a07f6d4e98bcc29db965ecd4f8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 02:53:42.635366 26222 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bd43f8a07f6d4e98bcc29db965ecd4f8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 02:53:42.653710 26665 tablet_bootstrap.cc:492] T bd43f8a07f6d4e98bcc29db965ecd4f8 P fae440a2b7cd48928353d69dc66c925e: Bootstrap starting.
I20260811 02:53:42.656476 26668 tablet_bootstrap.cc:492] T bd43f8a07f6d4e98bcc29db965ecd4f8 P 64c6f3fa2b7d4d7a80158a86e4e8e736: Bootstrap starting.
I20260811 02:53:42.656550 26665 tablet_bootstrap.cc:654] T bd43f8a07f6d4e98bcc29db965ecd4f8 P fae440a2b7cd48928353d69dc66c925e: Neither blocks nor log segments found. Creating new log.
I20260811 02:53:42.658015 26665 log.cc:824] T bd43f8a07f6d4e98bcc29db965ecd4f8 P fae440a2b7cd48928353d69dc66c925e: Log is configured to *not* fsync() on all Append() calls
I20260811 02:53:42.658526 26667 tablet_bootstrap.cc:492] T bd43f8a07f6d4e98bcc29db965ecd4f8 P c257dbb63ff84888891d41f05882f5e1: Bootstrap starting.
I20260811 02:53:42.664309 26667 tablet_bootstrap.cc:654] T bd43f8a07f6d4e98bcc29db965ecd4f8 P c257dbb63ff84888891d41f05882f5e1: Neither blocks nor log segments found. Creating new log.
I20260811 02:53:42.665436 26668 tablet_bootstrap.cc:654] T bd43f8a07f6d4e98bcc29db965ecd4f8 P 64c6f3fa2b7d4d7a80158a86e4e8e736: Neither blocks nor log segments found. Creating new log.
I20260811 02:53:42.666724 26667 log.cc:824] T bd43f8a07f6d4e98bcc29db965ecd4f8 P c257dbb63ff84888891d41f05882f5e1: Log is configured to *not* fsync() on all Append() calls
I20260811 02:53:42.667861 26668 log.cc:824] T bd43f8a07f6d4e98bcc29db965ecd4f8 P 64c6f3fa2b7d4d7a80158a86e4e8e736: Log is configured to *not* fsync() on all Append() calls
I20260811 02:53:42.671657 26668 tablet_bootstrap.cc:492] T bd43f8a07f6d4e98bcc29db965ecd4f8 P 64c6f3fa2b7d4d7a80158a86e4e8e736: No bootstrap required, opened a new log
I20260811 02:53:42.671993 26668 ts_tablet_manager.cc:1397] T bd43f8a07f6d4e98bcc29db965ecd4f8 P 64c6f3fa2b7d4d7a80158a86e4e8e736: Time spent bootstrapping tablet: real 0.016s user 0.005s sys 0.002s
I20260811 02:53:42.675369 26665 tablet_bootstrap.cc:492] T bd43f8a07f6d4e98bcc29db965ecd4f8 P fae440a2b7cd48928353d69dc66c925e: No bootstrap required, opened a new log
I20260811 02:53:42.675725 26665 ts_tablet_manager.cc:1397] T bd43f8a07f6d4e98bcc29db965ecd4f8 P fae440a2b7cd48928353d69dc66c925e: Time spent bootstrapping tablet: real 0.022s user 0.007s sys 0.000s
I20260811 02:53:42.678629 26668 raft_consensus.cc:348] T bd43f8a07f6d4e98bcc29db965ecd4f8 P 64c6f3fa2b7d4d7a80158a86e4e8e736 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64c6f3fa2b7d4d7a80158a86e4e8e736" member_type: VOTER last_known_addr { host: "127.12.138.195" port: 34729 } } peers { permanent_uuid: "fae440a2b7cd48928353d69dc66c925e" member_type: VOTER last_known_addr { host: "127.12.138.194" port: 33951 } } peers { permanent_uuid: "c257dbb63ff84888891d41f05882f5e1" member_type: VOTER last_known_addr { host: "127.12.138.193" port: 46843 } }
I20260811 02:53:42.680351 26665 raft_consensus.cc:348] T bd43f8a07f6d4e98bcc29db965ecd4f8 P fae440a2b7cd48928353d69dc66c925e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64c6f3fa2b7d4d7a80158a86e4e8e736" member_type: VOTER last_known_addr { host: "127.12.138.195" port: 34729 } } peers { permanent_uuid: "fae440a2b7cd48928353d69dc66c925e" member_type: VOTER last_known_addr { host: "127.12.138.194" port: 33951 } } peers { permanent_uuid: "c257dbb63ff84888891d41f05882f5e1" member_type: VOTER last_known_addr { host: "127.12.138.193" port: 46843 } }
I20260811 02:53:42.680817 26665 raft_consensus.cc:374] T bd43f8a07f6d4e98bcc29db965ecd4f8 P fae440a2b7cd48928353d69dc66c925e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 02:53:42.680917 26665 raft_consensus.cc:729] T bd43f8a07f6d4e98bcc29db965ecd4f8 P fae440a2b7cd48928353d69dc66c925e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fae440a2b7cd48928353d69dc66c925e, State: Initialized, Role: FOLLOWER
I20260811 02:53:42.680977 26667 tablet_bootstrap.cc:492] T bd43f8a07f6d4e98bcc29db965ecd4f8 P c257dbb63ff84888891d41f05882f5e1: No bootstrap required, opened a new log
I20260811 02:53:42.681365 26667 ts_tablet_manager.cc:1397] T bd43f8a07f6d4e98bcc29db965ecd4f8 P c257dbb63ff84888891d41f05882f5e1: Time spent bootstrapping tablet: real 0.023s user 0.007s sys 0.000s
I20260811 02:53:42.681511 26665 consensus_queue.cc:260] T bd43f8a07f6d4e98bcc29db965ecd4f8 P fae440a2b7cd48928353d69dc66c925e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64c6f3fa2b7d4d7a80158a86e4e8e736" member_type: VOTER last_known_addr { host: "127.12.138.195" port: 34729 } } peers { permanent_uuid: "fae440a2b7cd48928353d69dc66c925e" member_type: VOTER last_known_addr { host: "127.12.138.194" port: 33951 } } peers { permanent_uuid: "c257dbb63ff84888891d41f05882f5e1" member_type: VOTER last_known_addr { host: "127.12.138.193" port: 46843 } }
I20260811 02:53:42.682724 26665 ts_tablet_manager.cc:1428] T bd43f8a07f6d4e98bcc29db965ecd4f8 P fae440a2b7cd48928353d69dc66c925e: Time spent starting tablet: real 0.007s user 0.001s sys 0.004s
I20260811 02:53:42.683194 26478 heartbeater.cc:503] Master 127.12.138.254:33515 was elected leader, sending a full tablet report...
I20260811 02:53:42.686671 26667 raft_consensus.cc:348] T bd43f8a07f6d4e98bcc29db965ecd4f8 P c257dbb63ff84888891d41f05882f5e1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64c6f3fa2b7d4d7a80158a86e4e8e736" member_type: VOTER last_known_addr { host: "127.12.138.195" port: 34729 } } peers { permanent_uuid: "fae440a2b7cd48928353d69dc66c925e" member_type: VOTER last_known_addr { host: "127.12.138.194" port: 33951 } } peers { permanent_uuid: "c257dbb63ff84888891d41f05882f5e1" member_type: VOTER last_known_addr { host: "127.12.138.193" port: 46843 } }
I20260811 02:53:42.687276 26668 raft_consensus.cc:374] T bd43f8a07f6d4e98bcc29db965ecd4f8 P 64c6f3fa2b7d4d7a80158a86e4e8e736 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 02:53:42.687417 26668 raft_consensus.cc:729] T bd43f8a07f6d4e98bcc29db965ecd4f8 P 64c6f3fa2b7d4d7a80158a86e4e8e736 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 64c6f3fa2b7d4d7a80158a86e4e8e736, State: Initialized, Role: FOLLOWER
I20260811 02:53:42.687862 26667 raft_consensus.cc:374] T bd43f8a07f6d4e98bcc29db965ecd4f8 P c257dbb63ff84888891d41f05882f5e1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 02:53:42.688015 26667 raft_consensus.cc:729] T bd43f8a07f6d4e98bcc29db965ecd4f8 P c257dbb63ff84888891d41f05882f5e1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c257dbb63ff84888891d41f05882f5e1, State: Initialized, Role: FOLLOWER
I20260811 02:53:42.688452 26668 consensus_queue.cc:260] T bd43f8a07f6d4e98bcc29db965ecd4f8 P 64c6f3fa2b7d4d7a80158a86e4e8e736 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64c6f3fa2b7d4d7a80158a86e4e8e736" member_type: VOTER last_known_addr { host: "127.12.138.195" port: 34729 } } peers { permanent_uuid: "fae440a2b7cd48928353d69dc66c925e" member_type: VOTER last_known_addr { host: "127.12.138.194" port: 33951 } } peers { permanent_uuid: "c257dbb63ff84888891d41f05882f5e1" member_type: VOTER last_known_addr { host: "127.12.138.193" port: 46843 } }
I20260811 02:53:42.690450 26667 consensus_queue.cc:260] T bd43f8a07f6d4e98bcc29db965ecd4f8 P c257dbb63ff84888891d41f05882f5e1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64c6f3fa2b7d4d7a80158a86e4e8e736" member_type: VOTER last_known_addr { host: "127.12.138.195" port: 34729 } } peers { permanent_uuid: "fae440a2b7cd48928353d69dc66c925e" member_type: VOTER last_known_addr { host: "127.12.138.194" port: 33951 } } peers { permanent_uuid: "c257dbb63ff84888891d41f05882f5e1" member_type: VOTER last_known_addr { host: "127.12.138.193" port: 46843 } }
I20260811 02:53:42.691932 26668 ts_tablet_manager.cc:1428] T bd43f8a07f6d4e98bcc29db965ecd4f8 P 64c6f3fa2b7d4d7a80158a86e4e8e736: Time spent starting tablet: real 0.020s user 0.006s sys 0.001s
I20260811 02:53:42.693074 26667 ts_tablet_manager.cc:1428] T bd43f8a07f6d4e98bcc29db965ecd4f8 P c257dbb63ff84888891d41f05882f5e1: Time spent starting tablet: real 0.012s user 0.007s sys 0.000s
I20260811 02:53:42.693630 26643 heartbeater.cc:503] Master 127.12.138.254:33515 was elected leader, sending a full tablet report...
I20260811 02:53:42.694025 26307 heartbeater.cc:503] Master 127.12.138.254:33515 was elected leader, sending a full tablet report...
W20260811 02:53:42.714787 26308 tablet.cc:2369] T bd43f8a07f6d4e98bcc29db965ecd4f8 P c257dbb63ff84888891d41f05882f5e1: Can't schedule compaction. Clean time has not been advanced past its initial value.
Aug 11 02:53:42 dist-test-slave-9gcw krb5kdc[25478](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416821, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.138.254@KRBTEST.COM for HTTP/127.12.138.212@KRBTEST.COM
W20260811 02:53:42.779920 26644 tablet.cc:2369] T bd43f8a07f6d4e98bcc29db965ecd4f8 P 64c6f3fa2b7d4d7a80158a86e4e8e736: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 02:53:42.856503 26480 tablet.cc:2369] T bd43f8a07f6d4e98bcc29db965ecd4f8 P fae440a2b7cd48928353d69dc66c925e: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 02:53:42.985229 26674 raft_consensus.cc:482] T bd43f8a07f6d4e98bcc29db965ecd4f8 P 64c6f3fa2b7d4d7a80158a86e4e8e736 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 02:53:42.985539 26674 raft_consensus.cc:504] T bd43f8a07f6d4e98bcc29db965ecd4f8 P 64c6f3fa2b7d4d7a80158a86e4e8e736 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64c6f3fa2b7d4d7a80158a86e4e8e736" member_type: VOTER last_known_addr { host: "127.12.138.195" port: 34729 } } peers { permanent_uuid: "fae440a2b7cd48928353d69dc66c925e" member_type: VOTER last_known_addr { host: "127.12.138.194" port: 33951 } } peers { permanent_uuid: "c257dbb63ff84888891d41f05882f5e1" member_type: VOTER last_known_addr { host: "127.12.138.193" port: 46843 } }
I20260811 02:53:42.987061 26674 leader_election.cc:288] T bd43f8a07f6d4e98bcc29db965ecd4f8 P 64c6f3fa2b7d4d7a80158a86e4e8e736 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fae440a2b7cd48928353d69dc66c925e (127.12.138.194:33951), c257dbb63ff84888891d41f05882f5e1 (127.12.138.193:46843)
I20260811 02:53:42.992980 26645 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:53:42.987408 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.138.193:46843 (local address 127.12.138.195:56803)
0811 02:53:42.987864 (+ 456us) negotiation.cc:107] Waiting for socket to connect
0811 02:53:42.987889 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0811 02:53:42.987999 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:53:42.988446 (+ 447us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:53:42.988451 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:53:42.988480 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 02:53:42.988644 (+ 164us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:53:42.988655 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:53:42.989933 (+ 1278us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:53:42.989938 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:53:42.991284 (+ 1346us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:53:42.991299 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:53:42.992467 (+ 1168us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:53:42.992471 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:53:42.992597 (+ 126us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:53:42.992628 (+ 31us) client_negotiation.cc:769] Sending connection context
0811 02:53:42.992726 (+ 98us) client_negotiation.cc:240] Negotiation successful
0811 02:53:42.992809 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":359}
I20260811 02:53:42.993697 26664 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:53:42.987863 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.138.195:56803 (local address 127.12.138.193:46843)
0811 02:53:42.988168 (+ 305us) server_negotiation.cc:206] Beginning negotiation
0811 02:53:42.988173 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 02:53:42.988189 (+ 16us) server_negotiation.cc:407] Connection header received
0811 02:53:42.988244 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:53:42.988247 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:53:42.988297 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:53:42.988386 (+ 89us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 02:53:42.988841 (+ 455us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:53:42.989752 (+ 911us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:53:42.991501 (+ 1749us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:53:42.992167 (+ 666us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:53:42.992318 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:53:42.992389 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0811 02:53:42.993456 (+ 1067us) server_negotiation.cc:299] Negotiation successful
0811 02:53:42.993553 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":218}
I20260811 02:53:42.994297 26662 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:53:42.988770 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.138.195:38733 (local address 127.12.138.194:33951)
0811 02:53:42.988896 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0811 02:53:42.988900 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 02:53:42.989475 (+ 575us) server_negotiation.cc:407] Connection header received
0811 02:53:42.989695 (+ 220us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:53:42.989699 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:53:42.989750 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:53:42.989800 (+ 50us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 02:53:42.990251 (+ 451us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:53:42.991052 (+ 801us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:53:42.992593 (+ 1541us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:53:42.993260 (+ 667us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:53:42.993360 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:53:42.993407 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0811 02:53:42.994087 (+ 680us) server_negotiation.cc:299] Negotiation successful
0811 02:53:42.994168 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260811 02:53:42.994491 26248 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bd43f8a07f6d4e98bcc29db965ecd4f8" candidate_uuid: "64c6f3fa2b7d4d7a80158a86e4e8e736" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c257dbb63ff84888891d41f05882f5e1" is_pre_election: true
I20260811 02:53:42.994930 26680 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:53:42.987419 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.138.194:33951 (local address 127.12.138.195:38733)
0811 02:53:42.989361 (+ 1942us) negotiation.cc:107] Waiting for socket to connect
0811 02:53:42.989376 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 02:53:42.989548 (+ 172us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:53:42.989944 (+ 396us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:53:42.989947 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:53:42.989964 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 02:53:42.990107 (+ 143us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:53:42.990114 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:53:42.991218 (+ 1104us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:53:42.991221 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:53:42.992440 (+ 1219us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:53:42.992452 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:53:42.993712 (+ 1260us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:53:42.993715 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:53:42.993813 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:53:42.993837 (+ 24us) client_negotiation.cc:769] Sending connection context
0811 02:53:42.994633 (+ 796us) client_negotiation.cc:240] Negotiation successful
0811 02:53:42.994771 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1879,"mutex_wait_us":1490,"thread_start_us":133,"threads_started":1}
I20260811 02:53:42.994894 26248 raft_consensus.cc:2454] T bd43f8a07f6d4e98bcc29db965ecd4f8 P c257dbb63ff84888891d41f05882f5e1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64c6f3fa2b7d4d7a80158a86e4e8e736 in term 0.
I20260811 02:53:42.995638 26510 leader_election.cc:302] T bd43f8a07f6d4e98bcc29db965ecd4f8 P 64c6f3fa2b7d4d7a80158a86e4e8e736 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64c6f3fa2b7d4d7a80158a86e4e8e736, c257dbb63ff84888891d41f05882f5e1; no voters:
I20260811 02:53:42.995713 26402 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bd43f8a07f6d4e98bcc29db965ecd4f8" candidate_uuid: "64c6f3fa2b7d4d7a80158a86e4e8e736" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fae440a2b7cd48928353d69dc66c925e" is_pre_election: true
I20260811 02:53:42.995980 26402 raft_consensus.cc:2454] T bd43f8a07f6d4e98bcc29db965ecd4f8 P fae440a2b7cd48928353d69dc66c925e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64c6f3fa2b7d4d7a80158a86e4e8e736 in term 0.
I20260811 02:53:42.996269 26674 raft_consensus.cc:2781] T bd43f8a07f6d4e98bcc29db965ecd4f8 P 64c6f3fa2b7d4d7a80158a86e4e8e736 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 02:53:42.996356 26674 raft_consensus.cc:482] T bd43f8a07f6d4e98bcc29db965ecd4f8 P 64c6f3fa2b7d4d7a80158a86e4e8e736 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 02:53:42.996398 26674 raft_consensus.cc:3037] T bd43f8a07f6d4e98bcc29db965ecd4f8 P 64c6f3fa2b7d4d7a80158a86e4e8e736 [term 0 FOLLOWER]: Advancing to term 1
I20260811 02:53:42.998011 26674 raft_consensus.cc:504] T bd43f8a07f6d4e98bcc29db965ecd4f8 P 64c6f3fa2b7d4d7a80158a86e4e8e736 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64c6f3fa2b7d4d7a80158a86e4e8e736" member_type: VOTER last_known_addr { host: "127.12.138.195" port: 34729 } } peers { permanent_uuid: "fae440a2b7cd48928353d69dc66c925e" member_type: VOTER last_known_addr { host: "127.12.138.194" port: 33951 } } peers { permanent_uuid: "c257dbb63ff84888891d41f05882f5e1" member_type: VOTER last_known_addr { host: "127.12.138.193" port: 46843 } }
I20260811 02:53:42.998397 26674 leader_election.cc:288] T bd43f8a07f6d4e98bcc29db965ecd4f8 P 64c6f3fa2b7d4d7a80158a86e4e8e736 [CANDIDATE]: Term 1 election: Requested vote from peers fae440a2b7cd48928353d69dc66c925e (127.12.138.194:33951), c257dbb63ff84888891d41f05882f5e1 (127.12.138.193:46843)
I20260811 02:53:42.998874 26248 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bd43f8a07f6d4e98bcc29db965ecd4f8" candidate_uuid: "64c6f3fa2b7d4d7a80158a86e4e8e736" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c257dbb63ff84888891d41f05882f5e1"
I20260811 02:53:42.999027 26248 raft_consensus.cc:3037] T bd43f8a07f6d4e98bcc29db965ecd4f8 P c257dbb63ff84888891d41f05882f5e1 [term 0 FOLLOWER]: Advancing to term 1
I20260811 02:53:42.999400 26402 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bd43f8a07f6d4e98bcc29db965ecd4f8" candidate_uuid: "64c6f3fa2b7d4d7a80158a86e4e8e736" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fae440a2b7cd48928353d69dc66c925e"
I20260811 02:53:42.999526 26402 raft_consensus.cc:3037] T bd43f8a07f6d4e98bcc29db965ecd4f8 P fae440a2b7cd48928353d69dc66c925e [term 0 FOLLOWER]: Advancing to term 1
I20260811 02:53:43.000530 26248 raft_consensus.cc:2454] T bd43f8a07f6d4e98bcc29db965ecd4f8 P c257dbb63ff84888891d41f05882f5e1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64c6f3fa2b7d4d7a80158a86e4e8e736 in term 1.
I20260811 02:53:43.000862 26402 raft_consensus.cc:2454] T bd43f8a07f6d4e98bcc29db965ecd4f8 P fae440a2b7cd48928353d69dc66c925e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64c6f3fa2b7d4d7a80158a86e4e8e736 in term 1.
I20260811 02:53:43.001101 26510 leader_election.cc:302] T bd43f8a07f6d4e98bcc29db965ecd4f8 P 64c6f3fa2b7d4d7a80158a86e4e8e736 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64c6f3fa2b7d4d7a80158a86e4e8e736, c257dbb63ff84888891d41f05882f5e1; no voters:
I20260811 02:53:43.001524 26674 raft_consensus.cc:2781] T bd43f8a07f6d4e98bcc29db965ecd4f8 P 64c6f3fa2b7d4d7a80158a86e4e8e736 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 02:53:43.001793 26674 raft_consensus.cc:686] T bd43f8a07f6d4e98bcc29db965ecd4f8 P 64c6f3fa2b7d4d7a80158a86e4e8e736 [term 1 LEADER]: Becoming Leader. State: Replica: 64c6f3fa2b7d4d7a80158a86e4e8e736, State: Running, Role: LEADER
I20260811 02:53:43.002167 26674 consensus_queue.cc:237] T bd43f8a07f6d4e98bcc29db965ecd4f8 P 64c6f3fa2b7d4d7a80158a86e4e8e736 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64c6f3fa2b7d4d7a80158a86e4e8e736" member_type: VOTER last_known_addr { host: "127.12.138.195" port: 34729 } } peers { permanent_uuid: "fae440a2b7cd48928353d69dc66c925e" member_type: VOTER last_known_addr { host: "127.12.138.194" port: 33951 } } peers { permanent_uuid: "c257dbb63ff84888891d41f05882f5e1" member_type: VOTER last_known_addr { host: "127.12.138.193" port: 46843 } }
I20260811 02:53:43.006251 26051 catalog_manager.cc:5495] T bd43f8a07f6d4e98bcc29db965ecd4f8 P 64c6f3fa2b7d4d7a80158a86e4e8e736 reported cstate change: term changed from 0 to 1, leader changed from <none> to 64c6f3fa2b7d4d7a80158a86e4e8e736 (127.12.138.195). New cstate: current_term: 1 leader_uuid: "64c6f3fa2b7d4d7a80158a86e4e8e736" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64c6f3fa2b7d4d7a80158a86e4e8e736" member_type: VOTER last_known_addr { host: "127.12.138.195" port: 34729 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fae440a2b7cd48928353d69dc66c925e" member_type: VOTER last_known_addr { host: "127.12.138.194" port: 33951 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c257dbb63ff84888891d41f05882f5e1" member_type: VOTER last_known_addr { host: "127.12.138.193" port: 46843 } health_report { overall_health: UNKNOWN } } }
I20260811 02:53:43.077378 26660 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:53:43.073207 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54150 (local address 127.12.138.195:34729)
0811 02:53:43.073383 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0811 02:53:43.073388 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 02:53:43.073407 (+ 19us) server_negotiation.cc:407] Connection header received
0811 02:53:43.073498 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:53:43.073501 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:53:43.073572 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:53:43.073691 (+ 119us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 02:53:43.073975 (+ 284us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:53:43.074752 (+ 777us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:53:43.075616 (+ 864us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:53:43.075947 (+ 331us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:53:43.076055 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:53:43.076336 (+ 281us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 02:53:43.076907 (+ 571us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 02:53:43.077031 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0811 02:53:43.077169 (+ 138us) server_negotiation.cc:299] Negotiation successful
0811 02:53:43.077234 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260811 02:53:43.083417 26248 raft_consensus.cc:1261] T bd43f8a07f6d4e98bcc29db965ecd4f8 P c257dbb63ff84888891d41f05882f5e1 [term 1 FOLLOWER]: Refusing update from remote peer 64c6f3fa2b7d4d7a80158a86e4e8e736: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 02:53:43.083415 26402 raft_consensus.cc:1261] T bd43f8a07f6d4e98bcc29db965ecd4f8 P fae440a2b7cd48928353d69dc66c925e [term 1 FOLLOWER]: Refusing update from remote peer 64c6f3fa2b7d4d7a80158a86e4e8e736: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 02:53:43.084204 26682 consensus_queue.cc:1035] T bd43f8a07f6d4e98bcc29db965ecd4f8 P 64c6f3fa2b7d4d7a80158a86e4e8e736 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fae440a2b7cd48928353d69dc66c925e" member_type: VOTER last_known_addr { host: "127.12.138.194" port: 33951 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 02:53:43.084409 26674 consensus_queue.cc:1035] T bd43f8a07f6d4e98bcc29db965ecd4f8 P 64c6f3fa2b7d4d7a80158a86e4e8e736 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c257dbb63ff84888891d41f05882f5e1" member_type: VOTER last_known_addr { host: "127.12.138.193" port: 46843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 02:53:43.092473 26695 mvcc.cc:205] Tried to move back new op lower bound from 7317163307342008320 to 7317163307020083200. Current Snapshot: MvccSnapshot[applied={T|T < 7317163307342008320}]
I20260811 02:53:43.092645 26696 mvcc.cc:205] Tried to move back new op lower bound from 7317163307342008320 to 7317163307020083200. Current Snapshot: MvccSnapshot[applied={T|T < 7317163307342008320}]
I20260811 02:53:43.100117 26702 mvcc.cc:205] Tried to move back new op lower bound from 7317163307342008320 to 7317163307020083200. Current Snapshot: MvccSnapshot[applied={T|T < 7317163307342008320}]
I20260811 02:53:43.103916 26052 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52512:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 02:53:43.104085 26052 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52512:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 02:53:43.106738 26052 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b58647aa611d497f9fd438e24ca8baee: Sending DeleteTablet for 3 replicas of tablet bd43f8a07f6d4e98bcc29db965ecd4f8
I20260811 02:53:43.108119 26564 tablet_service.cc:1514] Processing DeleteTablet for tablet bd43f8a07f6d4e98bcc29db965ecd4f8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 02:53:43 UTC) from {username='kudu', principal='kudu/127.12.138.254@KRBTEST.COM'} at 127.0.0.1:54144
I20260811 02:53:43.108565 26399 tablet_service.cc:1514] Processing DeleteTablet for tablet bd43f8a07f6d4e98bcc29db965ecd4f8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 02:53:43 UTC) from {username='kudu', principal='kudu/127.12.138.254@KRBTEST.COM'} at 127.0.0.1:39718
I20260811 02:53:43.108968 26222 tablet_service.cc:1514] Processing DeleteTablet for tablet bd43f8a07f6d4e98bcc29db965ecd4f8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 02:53:43 UTC) from {username='kudu', principal='kudu/127.12.138.254@KRBTEST.COM'} at 127.0.0.1:37740
I20260811 02:53:43.110381 12843 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDawEwS/build/debug/bin/kudu with pid 26145
I20260811 02:53:43.118297 26709 tablet_replica.cc:330] T bd43f8a07f6d4e98bcc29db965ecd4f8 P fae440a2b7cd48928353d69dc66c925e: stopping tablet replica
I20260811 02:53:43.120122 26708 tablet_replica.cc:330] T bd43f8a07f6d4e98bcc29db965ecd4f8 P 64c6f3fa2b7d4d7a80158a86e4e8e736: stopping tablet replica
I20260811 02:53:43.120635 26709 raft_consensus.cc:2229] T bd43f8a07f6d4e98bcc29db965ecd4f8 P fae440a2b7cd48928353d69dc66c925e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 02:53:43.121011 26708 raft_consensus.cc:2229] T bd43f8a07f6d4e98bcc29db965ecd4f8 P 64c6f3fa2b7d4d7a80158a86e4e8e736 [term 1 LEADER]: Raft consensus shutting down.
I20260811 02:53:43.121627 26708 raft_consensus.cc:2258] T bd43f8a07f6d4e98bcc29db965ecd4f8 P 64c6f3fa2b7d4d7a80158a86e4e8e736 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 02:53:43.122040 26709 raft_consensus.cc:2258] T bd43f8a07f6d4e98bcc29db965ecd4f8 P fae440a2b7cd48928353d69dc66c925e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 02:53:43.124223 26709 ts_tablet_manager.cc:1905] T bd43f8a07f6d4e98bcc29db965ecd4f8 P fae440a2b7cd48928353d69dc66c925e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 02:53:43.132015 26708 ts_tablet_manager.cc:1905] T bd43f8a07f6d4e98bcc29db965ecd4f8 P 64c6f3fa2b7d4d7a80158a86e4e8e736: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 02:53:43.134042 26709 ts_tablet_manager.cc:1918] T bd43f8a07f6d4e98bcc29db965ecd4f8 P fae440a2b7cd48928353d69dc66c925e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 02:53:43.134307 26709 log.cc:1196] T bd43f8a07f6d4e98bcc29db965ecd4f8 P fae440a2b7cd48928353d69dc66c925e: Deleting WAL directory at /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786416707705494-12843-0/minicluster-data/ts-1/wal/wals/bd43f8a07f6d4e98bcc29db965ecd4f8
I20260811 02:53:43.134835 26709 ts_tablet_manager.cc:1939] T bd43f8a07f6d4e98bcc29db965ecd4f8 P fae440a2b7cd48928353d69dc66c925e: Deleting consensus metadata
I20260811 02:53:43.135821 26708 ts_tablet_manager.cc:1918] T bd43f8a07f6d4e98bcc29db965ecd4f8 P 64c6f3fa2b7d4d7a80158a86e4e8e736: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 02:53:43.136039 26708 log.cc:1196] T bd43f8a07f6d4e98bcc29db965ecd4f8 P 64c6f3fa2b7d4d7a80158a86e4e8e736: Deleting WAL directory at /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786416707705494-12843-0/minicluster-data/ts-2/wal/wals/bd43f8a07f6d4e98bcc29db965ecd4f8
I20260811 02:53:43.136526 26708 ts_tablet_manager.cc:1939] T bd43f8a07f6d4e98bcc29db965ecd4f8 P 64c6f3fa2b7d4d7a80158a86e4e8e736: Deleting consensus metadata
I20260811 02:53:43.136770 26034 catalog_manager.cc:4841] TS fae440a2b7cd48928353d69dc66c925e (127.12.138.194:33951): tablet bd43f8a07f6d4e98bcc29db965ecd4f8 (table test-table [id=2367bd72f7cd44d6a5727d7c3ef3dbc0]) successfully deleted
W20260811 02:53:43.137639 26035 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.12.138.193:46843 (error 108)
I20260811 02:53:43.138072 26034 catalog_manager.cc:4841] TS 64c6f3fa2b7d4d7a80158a86e4e8e736 (127.12.138.195:34729): tablet bd43f8a07f6d4e98bcc29db965ecd4f8 (table test-table [id=2367bd72f7cd44d6a5727d7c3ef3dbc0]) successfully deleted
I20260811 02:53:43.139187 12843 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDawEwS/build/debug/bin/kudu with pid 26315
I20260811 02:53:43.147866 26712 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:53:43.139578 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.138.193:46843 (local address 127.0.0.1:37744)
0811 02:53:43.147636 (+ 8058us) negotiation.cc:107] Waiting for socket to connect
0811 02:53:43.147738 (+ 102us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.138.193:46843: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":7951,"thread_start_us":65,"threads_started":1}
W20260811 02:53:43.148267 26035 catalog_manager.cc:4584] TS c257dbb63ff84888891d41f05882f5e1 (127.12.138.193:46843): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet bd43f8a07f6d4e98bcc29db965ecd4f8: Network error: Client connection negotiation failed: client connection to 127.12.138.193:46843: connect: Connection refused (error 111)
I20260811 02:53:43.151021 12843 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDawEwS/build/debug/bin/kudu with pid 26484
I20260811 02:53:43.159044 12843 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDawEwS/build/debug/bin/kudu with pid 26012
I20260811 02:53:43.170348 12843 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260811 02:53:43.515280 12843 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260811 02:53:43.515477 12843 mini_ranger.cc:66] Stopping Ranger...
I20260811 02:53:43.862768 12843 mini_ranger.cc:68] Stopped Ranger
2026-08-11 02:53:43.863 UTC [25517] LOG: received smart shutdown request
2026-08-11 02:53:43.867 UTC [25517] LOG: background worker "logical replication launcher" (PID 25529) exited with exit code 1
2026-08-11 02:53:43.867 UTC [25522] LOG: shutting down
2026-08-11 02:53:44.069 UTC [25517] LOG: database system is shut down
2026-08-11T02:53:44Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegration (51629 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers
Loading random data
Initializing database '/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786416707705494-12843-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 02:53:44 dist-test-slave-9gcw krb5kdc[26726](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 02:53:44 dist-test-slave-9gcw krb5kdc[26726](info): set up 2 sockets
Aug 11 02:53:44 dist-test-slave-9gcw krb5kdc[26726](info): commencing operation
krb5kdc: starting...
W20260811 02:53:45.337625 12843 mini_kdc.cc:121] Time spent starting KDC: real 1.100s user 0.004s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786416707705494-12843-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786416707705494-12843-0/krb5kdc/test-user.keytab.
Aug 11 02:53:45 dist-test-slave-9gcw krb5kdc[26726](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416825, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T02:53:45Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T02:53:45Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.12.138.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.12.138.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.12.138.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786416707705494-12843-0/krb5kdc/rangeradmin_127.12.138.212@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.12.138.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786416707705494-12843-0/krb5kdc/rangeradmin_127.12.138.212@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.12.138.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.12.138.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.12.138.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786416707705494-12843-0/krb5kdc/rangerlookup_127.12.138.212@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.12.138.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786416707705494-12843-0/krb5kdc/rangerlookup_127.12.138.212@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.12.138.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.12.138.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.12.138.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786416707705494-12843-0/krb5kdc/HTTP_127.12.138.212@KRBTEST.COM.keytab.
Entry for principal HTTP/127.12.138.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786416707705494-12843-0/krb5kdc/HTTP_127.12.138.212@KRBTEST.COM.keytab.
I20260811 02:53:45.540086 12843 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786416707705494-12843-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskDawEwS/build/debug/bin/postgres/pg_ctl -D /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786416707705494-12843-0/minicluster-data/postgres -l logfile start
2026-08-11 02:53:48.277 UTC [26764] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-08-11 02:53:48.278 UTC [26764] LOG: listening on IPv4 address "127.12.138.212", port 35249
2026-08-11 02:53:48.282 UTC [26764] LOG: listening on Unix socket "/tmp/.s.PGSQL.35249"
2026-08-11 02:53:48.305 UTC [26767] LOG: database system was shut down at 2026-08-11 02:53:46 UTC
2026-08-11 02:53:48.309 UTC [26764] LOG: database system is ready to accept connections
I20260811 02:53:49.395051 12843 mini_postgres.cc:96] Postgres bound to 35249
2026-08-11 02:53:49.402 UTC [26775] FATAL: database "slave" does not exist
127.12.138.212:35249 - accepting connections
I20260811 02:53:49.403319 12843 mini_ranger.cc:161] Starting Ranger...
I20260811 02:53:49.419673 12843 mini_ranger.cc:84] Created miniranger Postgres user
I20260811 02:53:49.941912 12843 mini_ranger.cc:87] Created ranger Postgres database
I20260811 02:53:49.941999 12843 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786416707705494-12843-0/minicluster-data/ranger-admin
2026-08-11 02:53:50,233 [I] DB FLAVOR :POSTGRES
2026-08-11 02:53:50,233 [I] --------- Verifying Ranger DB connection ---------
2026-08-11 02:53:50,233 [I] Checking connection..
2026-08-11 02:53:50,233 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDawEwS/build/debug/bin/postgresql.jar:/tmp/dist-test-taskDawEwS/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.12.138.212:35249/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-08-11 02:53:50,524 [I] Checking connection passed.
2026-08-11 02:53:50,524 [I] --------- Verifying version history table ---------
2026-08-11 02:53:50,525 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDawEwS/build/debug/bin/postgresql.jar:/tmp/dist-test-taskDawEwS/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.12.138.212:35249/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-11 02:53:50,764 [I] Table x_db_version_h does not exist in database ranger
2026-08-11 02:53:50,765 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDawEwS/build/debug/bin/postgresql.jar:/tmp/dist-test-taskDawEwS/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.12.138.212:35249/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-11 02:53:51,052 [I] Table x_db_version_h does not exist in database ranger
2026-08-11 02:53:51,052 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-08-11 02:53:51,052 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDawEwS/build/debug/bin/postgresql.jar:/tmp/dist-test-taskDawEwS/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.12.138.212:35249/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskDawEwS/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-08-11 02:53:51.333 UTC [26854] WARNING: there is no transaction in progress
2026-08-11 02:53:51,341 [I] create_dbversion_catalog.sql file imported successfully
2026-08-11 02:53:51,341 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDawEwS/build/debug/bin/postgresql.jar:/tmp/dist-test-taskDawEwS/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.12.138.212:35249/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-11 02:53:51,626 [I] Table x_db_version_h already exists in database 'ranger'
2026-08-11 02:53:51,626 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-11 02:53:51,627 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDawEwS/build/debug/bin/postgresql.jar:/tmp/dist-test-taskDawEwS/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.12.138.212:35249/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-08-11 02:53:51,893 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDawEwS/build/debug/bin/postgresql.jar:/tmp/dist-test-taskDawEwS/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.12.138.212:35249/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-08-11 02:53:52,165 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDawEwS/build/debug/bin/postgresql.jar:/tmp/dist-test-taskDawEwS/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.12.138.212:35249/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-9gcw.c.gcp-upstream.internal','N') ;"
2026-08-11 02:53:52,431 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDawEwS/build/debug/bin/postgresql.jar:/tmp/dist-test-taskDawEwS/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.12.138.212:35249/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-11 02:53:52,690 [I] Table x_portal_user does not exist in database ranger
2026-08-11 02:53:52,691 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDawEwS/build/debug/bin/postgresql.jar:/tmp/dist-test-taskDawEwS/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.12.138.212:35249/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-11 02:53:52,968 [I] Table x_policy_ref_group does not exist in database ranger
2026-08-11 02:53:52,969 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDawEwS/build/debug/bin/postgresql.jar:/tmp/dist-test-taskDawEwS/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.12.138.212:35249/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-11 02:53:53,209 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDawEwS/build/debug/bin/postgresql.jar:/tmp/dist-test-taskDawEwS/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.12.138.212:35249/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-11 02:53:53,470 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-08-11 02:53:53,470 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDawEwS/build/debug/bin/postgresql.jar:/tmp/dist-test-taskDawEwS/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.12.138.212:35249/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskDawEwS/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-08-11 02:53:54.575 UTC [26979] WARNING: there is no transaction in progress
2026-08-11 02:53:54.593 UTC [26979] WARNING: there is no transaction in progress
2026-08-11 02:53:54.611 UTC [26979] WARNING: there is no transaction in progress
2026-08-11 02:53:54.629 UTC [26979] WARNING: there is no transaction in progress
2026-08-11 02:53:54.646 UTC [26979] WARNING: there is no transaction in progress
2026-08-11 02:53:54.667 UTC [26979] WARNING: there is no transaction in progress
2026-08-11 02:53:54.724 UTC [26979] WARNING: there is no transaction in progress
2026-08-11 02:53:54.731 UTC [26979] WARNING: there is no transaction in progress
2026-08-11 02:53:54.747 UTC [26979] WARNING: there is no transaction in progress
2026-08-11 02:53:54.758 UTC [26979] WARNING: there is no transaction in progress
2026-08-11 02:53:54.773 UTC [26979] WARNING: there is no transaction in progress
2026-08-11 02:53:54.790 UTC [26979] WARNING: there is no transaction in progress
2026-08-11 02:53:54.800 UTC [26979] WARNING: there is no transaction in progress
2026-08-11 02:53:54.808 UTC [26979] WARNING: there is no transaction in progress
2026-08-11 02:53:54.815 UTC [26979] WARNING: there is no transaction in progress
2026-08-11 02:53:54.823 UTC [26979] WARNING: there is no transaction in progress
2026-08-11 02:53:55,539 [I] ranger_core_db_postgres.sql file imported successfully
2026-08-11 02:53:55,539 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDawEwS/build/debug/bin/postgresql.jar:/tmp/dist-test-taskDawEwS/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.12.138.212:35249/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-08-11 02:53:55,806 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-08-11 02:53:55,806 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDawEwS/build/debug/bin/postgresql.jar:/tmp/dist-test-taskDawEwS/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.12.138.212:35249/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-11 02:53:56,086 [I] Table x_portal_user already exists in database 'ranger'
2026-08-11 02:53:56,086 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDawEwS/build/debug/bin/postgresql.jar:/tmp/dist-test-taskDawEwS/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.12.138.212:35249/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-11 02:53:56,377 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-08-11 02:53:56,377 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDawEwS/build/debug/bin/postgresql.jar:/tmp/dist-test-taskDawEwS/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.12.138.212:35249/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-11 02:53:56,631 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDawEwS/build/debug/bin/postgresql.jar:/tmp/dist-test-taskDawEwS/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.12.138.212:35249/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-11 02:53:56,896 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDawEwS/build/debug/bin/postgresql.jar:/tmp/dist-test-taskDawEwS/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.12.138.212:35249/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-9gcw.c.gcp-upstream.internal';"
2026-08-11 02:53:57,187 [I] CORE_DB_SCHEMA import status has been updated
2026-08-11 02:53:57,188 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDawEwS/build/debug/bin/postgresql.jar:/tmp/dist-test-taskDawEwS/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.12.138.212:35249/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-08-11 02:53:57,449 [I] DB_PATCHES have already been applied
I20260811 02:53:57.455073 12843 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786416707705494-12843-0/minicluster-data/ranger-admin:/tmp/dist-test-taskDawEwS/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskDawEwS/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskDawEwS/build/debug/bin/postgresql.jar:/tmp/dist-test-taskDawEwS/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260811 02:53:57.455147 12843 mini_ranger.cc:193] Using host: 127.12.138.212
I20260811 02:53:57.459695 12843 mini_ranger.cc:227] Ranger admin URL: http://127.12.138.212:48515
Aug 11, 2026 2:53:57 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 11, 2026 2:53:57 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 11, 2026 2:53:58 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786416707705494-12843-0/minicluster-data/ranger-admin/ews/webapp
Aug 11, 2026 2:53:58 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786416707705494-12843-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Aug 11, 2026 2:53:58 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786416707705494-12843-0/minicluster-data/ranger-admin/ews/webapp] .....
Aug 11, 2026 2:53:58 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786416707705494-12843-0/minicluster-data/ranger-admin/ews/webapp].
Aug 11, 2026 2:53:58 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260811 02:53:58.743350 12843 mini_ranger.cc:160] Time spent starting Ranger: real 9.340s user 0.000s sys 0.013s
Aug 11, 2026 2:54:20 AM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Aug 11, 2026 2:54:20 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.UserREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
Aug 11, 2026 2:54:20 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Aug 11, 2026 2:54:20 AM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Aug 11, 2026 2:54:20 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Aug 11, 2026 2:54:20 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Aug 11, 2026 2:54:20 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Aug 11, 2026 2:54:20 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Aug 11, 2026 2:54:20 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Aug 11, 2026 2:54:20 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Aug 11, 2026 2:54:20 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Aug 11, 2026 2:54:20 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Aug 11, 2026 2:54:20 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Aug 11, 2026 2:54:20 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Aug 11, 2026 2:54:20 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Aug 11, 2026 2:54:20 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Aug 11, 2026 2:54:20 AM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Aug 11, 2026 2:54:21 AM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260811 02:54:21.981081 12843 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.12.138.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.12.138.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.12.138.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786416707705494-12843-0/krb5kdc/rangerkms_127.12.138.212@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.12.138.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786416707705494-12843-0/krb5kdc/rangerkms_127.12.138.212@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.12.138.212@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.12.138.212@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.12.138.212@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786416707705494-12843-0/krb5kdc/HTTP_127.12.138.212@KRBTEST.COM.keytab.
Entry for principal HTTP/127.12.138.212@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786416707705494-12843-0/krb5kdc/HTTP_127.12.138.212@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Aug 11 02:54:22 dist-test-slave-9gcw krb5kdc[26726](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416862, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260811 02:54:22.081482 12843 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260811 02:54:22.092015 12843 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260811 02:54:23.050484 12843 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260811 02:54:23.050587 12843 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786416707705494-12843-0/minicluster-data/ranger-kms
I20260811 02:54:23.050613 12843 mini_ranger_kms.cc:227] Using postgres at 127.12.138.212:35249
2026-08-11 02:54:23,249 [I] DB FLAVOR :POSTGRES
2026-08-11 02:54:23,249 [I] --------- Verifying Ranger DB connection ---------
2026-08-11 02:54:23,249 [I] Checking connection
2026-08-11 02:54:23,249 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDawEwS/build/debug/bin/postgresql.jar:/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786416707705494-12843-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.12.138.212:35249/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-08-11 02:54:23,501 [I] connection success
2026-08-11 02:54:23,501 [I] --------- Verifying Ranger DB tables ---------
2026-08-11 02:54:23,501 [I] Verifying table ranger_masterkey in database rangerkms
2026-08-11 02:54:23,501 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDawEwS/build/debug/bin/postgresql.jar:/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786416707705494-12843-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.12.138.212:35249/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-08-11 02:54:23,790 [I] Table ranger_masterkey does not exist in database rangerkms
2026-08-11 02:54:23,790 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-11 02:54:23,790 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-08-11 02:54:23,790 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDawEwS/build/debug/bin/postgresql.jar:/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786416707705494-12843-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.12.138.212:35249/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786416707705494-12843-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-08-11 02:54:24,063 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260811 02:54:24.642287 12843 mini_ranger_kms.cc:310] Created kms service
I20260811 02:54:24.861743 12843 mini_ranger_kms.cc:326] Created kudu user
I20260811 02:54:24.916852 12843 mini_ranger_kms.cc:343] Created rangerkms user
Aug 11 02:54:24 dist-test-slave-9gcw krb5kdc[26726](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416862, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.12.138.212@KRBTEST.COM
I20260811 02:54:25.357401 12843 mini_ranger_kms.cc:384] Added ranger policy
I20260811 02:54:25.357561 12843 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786416707705494-12843-0/minicluster-data/ranger-kms:/tmp/dist-test-taskDawEwS/build/debug/bin/postgresql.jar:/tmp/dist-test-taskDawEwS/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskDawEwS/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260811 02:54:25.357582 12843 mini_ranger_kms.cc:240] Using host: 127.12.138.212
I20260811 02:54:25.363008 12843 mini_ranger_kms.cc:276] Ranger KMS PID: 27189
I20260811 02:54:25.363114 12843 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.12.138.212:36171
Aug 11, 2026 2:54:26 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 11, 2026 2:54:26 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 11, 2026 2:54:26 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786416707705494-12843-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Aug 11, 2026 2:54:26 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786416707705494-12843-0/minicluster-data/ranger-kms/ews/webapp] .....
Aug 11, 2026 2:54:27 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786416707705494-12843-0/minicluster-data/ranger-kms/ews/webapp].
Aug 11, 2026 2:54:27 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
I20260811 02:54:27.909696 12843 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 5.828s user 0.009s sys 0.171s
I20260811 02:54:27.909847 12843 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260811 02:54:27.909925 12843 mini_ranger_kms.cc:401] 127.12.138.212:36171/kms/v1/keys
Aug 11 02:54:28 dist-test-slave-9gcw krb5kdc[26726](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786416868, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.12.138.212@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Aug 11 02:54:28 dist-test-slave-9gcw krb5kdc[26726](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416868, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.12.138.212@KRBTEST.COM for HTTP/127.12.138.212@KRBTEST.COM
Aug 11 02:54:28 dist-test-slave-9gcw krb5kdc[26726](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416868, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.12.138.212@KRBTEST.COM for HTTP/127.12.138.212@KRBTEST.COM
Aug 11 02:54:31 dist-test-slave-9gcw krb5kdc[26726](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416871, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.12.138.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.138.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.138.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786416707705494-12843-0/krb5kdc/kudu_127.12.138.254.keytab.
Entry for principal kudu/127.12.138.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786416707705494-12843-0/krb5kdc/kudu_127.12.138.254.keytab.
I20260811 02:54:31.156930 12843 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDawEwS/build/debug/bin/kudu
/tmp/dist-test-taskDawEwS/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786416707705494-12843-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786416707705494-12843-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786416707705494-12843-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786416707705494-12843-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.138.254:35723
--webserver_interface=127.12.138.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786416707705494-12843-0/krb5kdc/kudu_127.12.138.254.keytab
--principal=kudu/127.12.138.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.138.212:46223
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.12.138.254:35723
--ranger_config_path=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786416707705494-12843-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.12.138.212:36171/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786416707705494-12843-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786416707705494-12843-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786416707705494-12843-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 02:54:31.288116 27249 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 02:54:31.288419 27249 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 02:54:31.288462 27249 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 02:54:31.292601 27249 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 02:54:31.292678 27249 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 02:54:31.292698 27249 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 02:54:31.292716 27249 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 02:54:31.292781 27249 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 02:54:31.298513 27249 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.138.212:46223
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786416707705494-12843-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786416707705494-12843-0/minicluster-data/master-0/wal
--ranger_kms_url=127.12.138.212:36171/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.12.138.254:35723
--ranger_config_path=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786416707705494-12843-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786416707705494-12843-0/krb5kdc/kudu_127.12.138.254.keytab
--principal=kudu/127.12.138.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.138.254:35723
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786416707705494-12843-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.12.138.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27249
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786416707705494-12843-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 02:09:49 UTC on ccycloud.quasar-fsnqqc.root.comops.site
build id 4579570
I20260811 02:54:31.299623 27249 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 02:54:31.300673 27249 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 02:54:31.310039 27258 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:54:31.310082 27255 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:54:31.310094 27256 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:54:31.310173 27249 server_base.cc:1034] running on GCE node
I20260811 02:54:31.310973 27249 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 02:54:31.312155 27249 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 02:54:31.313339 27249 hybrid_clock.cc:648] HybridClock initialized: now 1786416871313324 us; error 28 us; skew 500 ppm
Aug 11 02:54:31 dist-test-slave-9gcw krb5kdc[26726](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416871, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.138.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 02:54:31.316283 27249 init.cc:377] Logged in from keytab as kudu/127.12.138.254@KRBTEST.COM (short username kudu)
I20260811 02:54:31.317247 27249 webserver.cc:460] Webserver started at http://127.12.138.254:38885/ using document root <none> and password file <none>
I20260811 02:54:31.317816 27249 fs_manager.cc:359] Metadata directory not provided
I20260811 02:54:31.317899 27249 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 02:54:31.318123 27249 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 11 02:54:31 dist-test-slave-9gcw krb5kdc[26726](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416871, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.138.254@KRBTEST.COM for HTTP/127.12.138.212@KRBTEST.COM
I20260811 02:54:31.366771 27249 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786416707705494-12843-0/minicluster-data/master-0/data/instance:
uuid: "f36ef9766adb417d848c4dcad0b4baf5"
format_stamp: "Formatted at 2026-08-11 02:54:31 on dist-test-slave-9gcw"
server_key: "a425925ced721729633d087deb9eb9ac"
server_key_iv: "08ae368e2c97f1ce68d9f758bc850ffb"
server_key_version: "kuduclusterkey@0"
I20260811 02:54:31.367440 27249 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786416707705494-12843-0/minicluster-data/master-0/wal/instance:
uuid: "f36ef9766adb417d848c4dcad0b4baf5"
format_stamp: "Formatted at 2026-08-11 02:54:31 on dist-test-slave-9gcw"
server_key: "a425925ced721729633d087deb9eb9ac"
server_key_iv: "08ae368e2c97f1ce68d9f758bc850ffb"
server_key_version: "kuduclusterkey@0"
I20260811 02:54:31.371800 27249 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260811 02:54:31.386732 27266 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 02:54:31.388267 27249 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260811 02:54:31.388451 27249 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786416707705494-12843-0/minicluster-data/master-0/data,/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786416707705494-12843-0/minicluster-data/master-0/wal
uuid: "f36ef9766adb417d848c4dcad0b4baf5"
format_stamp: "Formatted at 2026-08-11 02:54:31 on dist-test-slave-9gcw"
server_key: "a425925ced721729633d087deb9eb9ac"
server_key_iv: "08ae368e2c97f1ce68d9f758bc850ffb"
server_key_version: "kuduclusterkey@0"
I20260811 02:54:31.388597 27249 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786416707705494-12843-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786416707705494-12843-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786416707705494-12843-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 02:54:31.412034 27249 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 02:54:31.415666 27249 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 02:54:31.415932 27249 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 02:54:31.425438 27249 rpc_server.cc:307] RPC server started. Bound to: 127.12.138.254:35723
I20260811 02:54:31.425441 27327 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.138.254:35723 every 8 connection(s)
I20260811 02:54:31.426700 27249 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786416707705494-12843-0/minicluster-data/master-0/data/info.pb
I20260811 02:54:31.429999 27328 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 02:54:31.436348 27328 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f36ef9766adb417d848c4dcad0b4baf5: Bootstrap starting.
I20260811 02:54:31.436769 12843 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDawEwS/build/debug/bin/kudu as pid 27249
I20260811 02:54:31.436913 12843 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786416707705494-12843-0/minicluster-data/master-0/wal/instance
I20260811 02:54:31.437335 12843 external_mini_cluster.cc:1428] Setting key 8e0fb876c7583d0349172257c1b49386
I20260811 02:54:31.439536 27328 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f36ef9766adb417d848c4dcad0b4baf5: Neither blocks nor log segments found. Creating new log.
I20260811 02:54:31.440387 27328 log.cc:824] T 00000000000000000000000000000000 P f36ef9766adb417d848c4dcad0b4baf5: Log is configured to *not* fsync() on all Append() calls
I20260811 02:54:31.442828 27328 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f36ef9766adb417d848c4dcad0b4baf5: No bootstrap required, opened a new log
I20260811 02:54:31.447979 27328 raft_consensus.cc:348] T 00000000000000000000000000000000 P f36ef9766adb417d848c4dcad0b4baf5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f36ef9766adb417d848c4dcad0b4baf5" member_type: VOTER last_known_addr { host: "127.12.138.254" port: 35723 } }
I20260811 02:54:31.448410 27328 raft_consensus.cc:374] T 00000000000000000000000000000000 P f36ef9766adb417d848c4dcad0b4baf5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 02:54:31.448491 27328 raft_consensus.cc:729] T 00000000000000000000000000000000 P f36ef9766adb417d848c4dcad0b4baf5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f36ef9766adb417d848c4dcad0b4baf5, State: Initialized, Role: FOLLOWER
Aug 11 02:54:31 dist-test-slave-9gcw krb5kdc[26726](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416871, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.12.138.254@KRBTEST.COM
I20260811 02:54:31.449091 27328 consensus_queue.cc:260] T 00000000000000000000000000000000 P f36ef9766adb417d848c4dcad0b4baf5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f36ef9766adb417d848c4dcad0b4baf5" member_type: VOTER last_known_addr { host: "127.12.138.254" port: 35723 } }
I20260811 02:54:31.449316 27328 raft_consensus.cc:388] T 00000000000000000000000000000000 P f36ef9766adb417d848c4dcad0b4baf5 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 02:54:31.449424 27328 raft_consensus.cc:482] T 00000000000000000000000000000000 P f36ef9766adb417d848c4dcad0b4baf5 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 02:54:31.449570 27328 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f36ef9766adb417d848c4dcad0b4baf5 [term 0 FOLLOWER]: Advancing to term 1
I20260811 02:54:31.450877 27328 raft_consensus.cc:504] T 00000000000000000000000000000000 P f36ef9766adb417d848c4dcad0b4baf5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f36ef9766adb417d848c4dcad0b4baf5" member_type: VOTER last_known_addr { host: "127.12.138.254" port: 35723 } }
I20260811 02:54:31.451308 27328 leader_election.cc:302] T 00000000000000000000000000000000 P f36ef9766adb417d848c4dcad0b4baf5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f36ef9766adb417d848c4dcad0b4baf5; no voters:
I20260811 02:54:31.451731 27328 leader_election.cc:288] T 00000000000000000000000000000000 P f36ef9766adb417d848c4dcad0b4baf5 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 02:54:31.451872 27333 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f36ef9766adb417d848c4dcad0b4baf5 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 02:54:31.452160 27333 raft_consensus.cc:686] T 00000000000000000000000000000000 P f36ef9766adb417d848c4dcad0b4baf5 [term 1 LEADER]: Becoming Leader. State: Replica: f36ef9766adb417d848c4dcad0b4baf5, State: Running, Role: LEADER
I20260811 02:54:31.452530 27333 consensus_queue.cc:237] T 00000000000000000000000000000000 P f36ef9766adb417d848c4dcad0b4baf5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f36ef9766adb417d848c4dcad0b4baf5" member_type: VOTER last_known_addr { host: "127.12.138.254" port: 35723 } }
I20260811 02:54:31.453279 27328 sys_catalog.cc:564] T 00000000000000000000000000000000 P f36ef9766adb417d848c4dcad0b4baf5 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 02:54:31.454016 27333 sys_catalog.cc:455] T 00000000000000000000000000000000 P f36ef9766adb417d848c4dcad0b4baf5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f36ef9766adb417d848c4dcad0b4baf5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f36ef9766adb417d848c4dcad0b4baf5" member_type: VOTER last_known_addr { host: "127.12.138.254" port: 35723 } } }
I20260811 02:54:31.454263 27333 sys_catalog.cc:458] T 00000000000000000000000000000000 P f36ef9766adb417d848c4dcad0b4baf5 [sys.catalog]: This master's current role is: LEADER
I20260811 02:54:31.454559 27333 sys_catalog.cc:455] T 00000000000000000000000000000000 P f36ef9766adb417d848c4dcad0b4baf5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f36ef9766adb417d848c4dcad0b4baf5. Latest consensus state: current_term: 1 leader_uuid: "f36ef9766adb417d848c4dcad0b4baf5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f36ef9766adb417d848c4dcad0b4baf5" member_type: VOTER last_known_addr { host: "127.12.138.254" port: 35723 } } }
I20260811 02:54:31.454725 27333 sys_catalog.cc:458] T 00000000000000000000000000000000 P f36ef9766adb417d848c4dcad0b4baf5 [sys.catalog]: This master's current role is: LEADER
I20260811 02:54:31.456072 27328 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786416707705494-12843-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260811 02:54:31.456112 27331 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:54:31.438883 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43486 (local address 127.12.138.254:35723)
0811 02:54:31.439429 (+ 546us) server_negotiation.cc:206] Beginning negotiation
0811 02:54:31.439452 (+ 23us) server_negotiation.cc:399] Waiting for connection header
0811 02:54:31.439497 (+ 45us) server_negotiation.cc:407] Connection header received
0811 02:54:31.440371 (+ 874us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:54:31.440395 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:54:31.440838 (+ 443us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:54:31.441245 (+ 407us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:54:31.442269 (+ 1024us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:54:31.443866 (+ 1597us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:54:31.445400 (+ 1534us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:54:31.446119 (+ 719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:54:31.446285 (+ 166us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:54:31.449694 (+ 3409us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:54:31.449701 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:54:31.449721 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:54:31.449757 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:54:31.452040 (+ 2283us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:54:31.452601 (+ 561us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:54:31.452607 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:54:31.452613 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:54:31.452760 (+ 147us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:54:31.453081 (+ 321us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:54:31.453084 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:54:31.453086 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:54:31.453641 (+ 555us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:54:31.454576 (+ 935us) server_negotiation.cc:1035] Waiting for connection context
0811 02:54:31.454748 (+ 172us) server_negotiation.cc:299] Negotiation successful
0811 02:54:31.455079 (+ 331us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":380,"thread_start_us":182,"threads_started":1}
I20260811 02:54:31.457723 27342 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 02:54:31.464735 27342 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 02:54:31.472488 27342 catalog_manager.cc:1347] Generated new cluster ID: 22c53b2e41254dc8b7511156533262aa
I20260811 02:54:31.472697 27342 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 02:54:31.481770 27342 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 02:54:31.483187 27342 catalog_manager.cc:1504] Loading token signing keys...
I20260811 02:54:31.493582 27342 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f36ef9766adb417d848c4dcad0b4baf5: Generated new TSK 0
I20260811 02:54:31.495038 27342 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.12.138.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.138.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.138.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786416707705494-12843-0/krb5kdc/kudu_127.12.138.193.keytab.
Entry for principal kudu/127.12.138.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786416707705494-12843-0/krb5kdc/kudu_127.12.138.193.keytab.
I20260811 02:54:32.250138 12843 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDawEwS/build/debug/bin/kudu
/tmp/dist-test-taskDawEwS/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786416707705494-12843-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786416707705494-12843-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786416707705494-12843-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786416707705494-12843-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.12.138.212:36171/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.138.193:0
--local_ip_for_outbound_sockets=127.12.138.193
--webserver_interface=127.12.138.193
--webserver_port=0
--tserver_master_addrs=127.12.138.254:35723
--keytab_file=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786416707705494-12843-0/krb5kdc/kudu_127.12.138.193.keytab
--principal=kudu/127.12.138.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.138.212:46223
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786416707705494-12843-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786416707705494-12843-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786416707705494-12843-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 02:54:32.386518 27377 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 02:54:32.386821 27377 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 02:54:32.386952 27377 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 02:54:32.391186 27377 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 02:54:32.391283 27377 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 02:54:32.391387 27377 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.138.193
I20260811 02:54:32.397032 27377 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.138.212:46223
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786416707705494-12843-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786416707705494-12843-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.12.138.212:36171/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786416707705494-12843-0/krb5kdc/kudu_127.12.138.193.keytab
--principal=kudu/127.12.138.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.138.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786416707705494-12843-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.12.138.193
--webserver_port=0
--tserver_master_addrs=127.12.138.254:35723
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27377
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.138.193
--log_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786416707705494-12843-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 02:09:49 UTC on ccycloud.quasar-fsnqqc.root.comops.site
build id 4579570
I20260811 02:54:32.398265 27377 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 02:54:32.399302 27377 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 02:54:32.410006 27386 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:54:32.410171 27385 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:54:32.409931 27388 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:54:32.410445 27377 server_base.cc:1034] running on GCE node
I20260811 02:54:32.410871 27377 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 02:54:32.411533 27377 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 02:54:32.412714 27377 hybrid_clock.cc:648] HybridClock initialized: now 1786416872412698 us; error 34 us; skew 500 ppm
Aug 11 02:54:32 dist-test-slave-9gcw krb5kdc[26726](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416872, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.138.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 02:54:32.415688 27377 init.cc:377] Logged in from keytab as kudu/127.12.138.193@KRBTEST.COM (short username kudu)
I20260811 02:54:32.416678 27377 webserver.cc:460] Webserver started at http://127.12.138.193:38755/ using document root <none> and password file <none>
I20260811 02:54:32.417356 27377 fs_manager.cc:359] Metadata directory not provided
I20260811 02:54:32.417454 27377 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 02:54:32.417690 27377 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 11 02:54:32 dist-test-slave-9gcw krb5kdc[26726](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416872, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.138.193@KRBTEST.COM for HTTP/127.12.138.212@KRBTEST.COM
I20260811 02:54:32.433185 27377 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786416707705494-12843-0/minicluster-data/ts-0/data/instance:
uuid: "d77db62fe6364395997bd0e8fd8adebc"
format_stamp: "Formatted at 2026-08-11 02:54:32 on dist-test-slave-9gcw"
server_key: "5376e966589ce934156857ff7b3a3aad"
server_key_iv: "e4f41f7764dba46c08d5839a283e88a3"
server_key_version: "kuduclusterkey@0"
I20260811 02:54:32.434149 27377 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786416707705494-12843-0/minicluster-data/ts-0/wal/instance:
uuid: "d77db62fe6364395997bd0e8fd8adebc"
format_stamp: "Formatted at 2026-08-11 02:54:32 on dist-test-slave-9gcw"
server_key: "5376e966589ce934156857ff7b3a3aad"
server_key_iv: "e4f41f7764dba46c08d5839a283e88a3"
server_key_version: "kuduclusterkey@0"
I20260811 02:54:32.446791 27377 fs_manager.cc:691] Time spent creating directory manager: real 0.012s user 0.004s sys 0.000s
I20260811 02:54:32.473769 27396 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 02:54:32.475827 27377 fs_manager.cc:725] Time spent opening block manager: real 0.013s user 0.002s sys 0.001s
I20260811 02:54:32.475997 27377 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786416707705494-12843-0/minicluster-data/ts-0/data,/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786416707705494-12843-0/minicluster-data/ts-0/wal
uuid: "d77db62fe6364395997bd0e8fd8adebc"
format_stamp: "Formatted at 2026-08-11 02:54:32 on dist-test-slave-9gcw"
server_key: "5376e966589ce934156857ff7b3a3aad"
server_key_iv: "e4f41f7764dba46c08d5839a283e88a3"
server_key_version: "kuduclusterkey@0"
I20260811 02:54:32.476168 27377 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786416707705494-12843-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786416707705494-12843-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786416707705494-12843-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 02:54:32.556173 27377 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 02:54:32.560290 27377 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 02:54:32.560513 27377 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 02:54:32.561342 27377 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 02:54:32.562611 27377 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 02:54:32.562682 27377 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 02:54:32.562741 27377 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 02:54:32.562820 27377 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.001s
Aug 11 02:54:32 dist-test-slave-9gcw krb5kdc[26726](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786416872, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.138.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 02:54:32.575310 27377 rpc_server.cc:307] RPC server started. Bound to: 127.12.138.193:46259
I20260811 02:54:32.576615 27377 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786416707705494-12843-0/minicluster-data/ts-0/data/info.pb
I20260811 02:54:32.581485 12843 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDawEwS/build/debug/bin/kudu as pid 27377
I20260811 02:54:32.581609 12843 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786416707705494-12843-0/minicluster-data/ts-0/wal/instance
I20260811 02:54:32.581965 12843 external_mini_cluster.cc:1428] Setting key 795cc34c72b6c31e3f427dd551101087
I20260811 02:54:32.579005 27530 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.138.193:46259 every 8 connection(s)
Aug 11 02:54:32 dist-test-slave-9gcw krb5kdc[26726](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416872, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.138.193@KRBTEST.COM for kudu/127.12.138.254@KRBTEST.COM
I20260811 02:54:32.599324 27534 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:54:32.581535 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.138.254:35723 (local address 127.12.138.193:46101)
0811 02:54:32.583318 (+ 1783us) negotiation.cc:107] Waiting for socket to connect
0811 02:54:32.583377 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0811 02:54:32.584550 (+ 1173us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:54:32.585859 (+ 1309us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:54:32.585870 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:54:32.586259 (+ 389us) client_negotiation.cc:189] Negotiated authn=SASL
0811 02:54:32.587032 (+ 773us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:54:32.587055 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:54:32.588520 (+ 1465us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:54:32.588524 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:54:32.589997 (+ 1473us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:54:32.590009 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:54:32.590728 (+ 719us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:54:32.590732 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:54:32.590947 (+ 215us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:54:32.591697 (+ 750us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 02:54:32.591720 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0811 02:54:32.593559 (+ 1839us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 02:54:32.595929 (+ 2370us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:54:32.595936 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:54:32.595953 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:54:32.596329 (+ 376us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:54:32.596658 (+ 329us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:54:32.596662 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:54:32.596664 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:54:32.596842 (+ 178us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:54:32.597358 (+ 516us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 02:54:32.597365 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 02:54:32.597684 (+ 319us) client_negotiation.cc:769] Sending connection context
0811 02:54:32.597876 (+ 192us) client_negotiation.cc:240] Negotiation successful
0811 02:54:32.598214 (+ 338us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1491,"thread_start_us":161,"threads_started":1}
I20260811 02:54:32.600126 27533 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:54:32.580981 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.138.193:46101 (local address 127.12.138.254:35723)
0811 02:54:32.585353 (+ 4372us) server_negotiation.cc:206] Beginning negotiation
0811 02:54:32.585358 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 02:54:32.585374 (+ 16us) server_negotiation.cc:407] Connection header received
0811 02:54:32.585424 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:54:32.585428 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:54:32.585490 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:54:32.587215 (+ 1725us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:54:32.587333 (+ 118us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:54:32.588315 (+ 982us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:54:32.590154 (+ 1839us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:54:32.590543 (+ 389us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:54:32.592604 (+ 2061us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:54:32.593783 (+ 1179us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:54:32.593789 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:54:32.593793 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:54:32.593824 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:54:32.595788 (+ 1964us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:54:32.596458 (+ 670us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:54:32.596462 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:54:32.596464 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:54:32.596554 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:54:32.596961 (+ 407us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:54:32.596965 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:54:32.596967 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:54:32.597213 (+ 246us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:54:32.599725 (+ 2512us) server_negotiation.cc:1035] Waiting for connection context
0811 02:54:32.599825 (+ 100us) server_negotiation.cc:299] Negotiation successful
0811 02:54:32.599953 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4287,"thread_start_us":104,"threads_started":1}
I20260811 02:54:32.601359 27531 heartbeater.cc:347] Connected to a master server at 127.12.138.254:35723
I20260811 02:54:32.601671 27531 heartbeater.cc:464] Registering TS with master...
I20260811 02:54:32.602306 27531 heartbeater.cc:511] Master 127.12.138.254:35723 requested a full tablet report, sending...
I20260811 02:54:32.604223 27284 ts_manager.cc:194] Registered new tserver with Master: d77db62fe6364395997bd0e8fd8adebc (127.12.138.193:46259)
I20260811 02:54:32.605933 27284 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.12.138.193@KRBTEST.COM'} at 127.12.138.193:46101
WARNING: no policy specified for kudu/127.12.138.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.138.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.138.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786416707705494-12843-0/krb5kdc/kudu_127.12.138.194.keytab.
Entry for principal kudu/127.12.138.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786416707705494-12843-0/krb5kdc/kudu_127.12.138.194.keytab.
I20260811 02:54:32.641310 12843 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDawEwS/build/debug/bin/kudu
/tmp/dist-test-taskDawEwS/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786416707705494-12843-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786416707705494-12843-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786416707705494-12843-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786416707705494-12843-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.12.138.212:36171/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.138.194:0
--local_ip_for_outbound_sockets=127.12.138.194
--webserver_interface=127.12.138.194
--webserver_port=0
--tserver_master_addrs=127.12.138.254:35723
--keytab_file=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786416707705494-12843-0/krb5kdc/kudu_127.12.138.194.keytab
--principal=kudu/127.12.138.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.138.212:46223
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786416707705494-12843-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786416707705494-12843-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786416707705494-12843-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 02:54:32.801366 27538 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 02:54:32.801676 27538 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 02:54:32.801753 27538 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 02:54:32.805950 27538 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 02:54:32.806044 27538 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 02:54:32.806137 27538 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.138.194
I20260811 02:54:32.811754 27538 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.138.212:46223
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786416707705494-12843-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786416707705494-12843-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.12.138.212:36171/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786416707705494-12843-0/krb5kdc/kudu_127.12.138.194.keytab
--principal=kudu/127.12.138.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.138.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786416707705494-12843-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.12.138.194
--webserver_port=0
--tserver_master_addrs=127.12.138.254:35723
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27538
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.138.194
--log_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786416707705494-12843-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 02:09:49 UTC on ccycloud.quasar-fsnqqc.root.comops.site
build id 4579570
I20260811 02:54:32.812918 27538 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 02:54:32.814044 27538 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 02:54:32.830762 27547 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:54:32.830770 27544 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:54:32.835808 27538 server_base.cc:1034] running on GCE node
W20260811 02:54:32.837347 27545 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:54:32.837982 27538 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 02:54:32.838866 27538 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 02:54:32.840040 27538 hybrid_clock.cc:648] HybridClock initialized: now 1786416872839999 us; error 61 us; skew 500 ppm
Aug 11 02:54:32 dist-test-slave-9gcw krb5kdc[26726](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416872, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.138.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 02:54:32.843974 27538 init.cc:377] Logged in from keytab as kudu/127.12.138.194@KRBTEST.COM (short username kudu)
I20260811 02:54:32.845216 27538 webserver.cc:460] Webserver started at http://127.12.138.194:37669/ using document root <none> and password file <none>
I20260811 02:54:32.845950 27538 fs_manager.cc:359] Metadata directory not provided
I20260811 02:54:32.846043 27538 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 02:54:32.846297 27538 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 11 02:54:32 dist-test-slave-9gcw krb5kdc[26726](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416872, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.138.194@KRBTEST.COM for HTTP/127.12.138.212@KRBTEST.COM
I20260811 02:54:32.862867 27538 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786416707705494-12843-0/minicluster-data/ts-1/data/instance:
uuid: "685778c8f32c4f4eae047d64c7907f89"
format_stamp: "Formatted at 2026-08-11 02:54:32 on dist-test-slave-9gcw"
server_key: "4d05b9be606378564458792f405011b4"
server_key_iv: "f14bc8f813b1e8767fbbf6c7c4aa731f"
server_key_version: "kuduclusterkey@0"
I20260811 02:54:32.864569 27538 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786416707705494-12843-0/minicluster-data/ts-1/wal/instance:
uuid: "685778c8f32c4f4eae047d64c7907f89"
format_stamp: "Formatted at 2026-08-11 02:54:32 on dist-test-slave-9gcw"
server_key: "4d05b9be606378564458792f405011b4"
server_key_iv: "f14bc8f813b1e8767fbbf6c7c4aa731f"
server_key_version: "kuduclusterkey@0"
I20260811 02:54:32.879020 27538 fs_manager.cc:691] Time spent creating directory manager: real 0.014s user 0.004s sys 0.000s
I20260811 02:54:32.912948 27555 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 02:54:32.914911 27538 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.002s sys 0.000s
I20260811 02:54:32.915067 27538 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786416707705494-12843-0/minicluster-data/ts-1/data,/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786416707705494-12843-0/minicluster-data/ts-1/wal
uuid: "685778c8f32c4f4eae047d64c7907f89"
format_stamp: "Formatted at 2026-08-11 02:54:32 on dist-test-slave-9gcw"
server_key: "4d05b9be606378564458792f405011b4"
server_key_iv: "f14bc8f813b1e8767fbbf6c7c4aa731f"
server_key_version: "kuduclusterkey@0"
I20260811 02:54:32.915223 27538 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786416707705494-12843-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786416707705494-12843-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786416707705494-12843-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 02:54:32.932732 27538 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 02:54:32.937062 27538 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 02:54:32.937443 27538 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 02:54:32.938301 27538 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 02:54:32.939517 27538 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 02:54:32.939636 27538 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 02:54:32.939785 27538 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 02:54:32.939865 27538 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 02:54:32.954560 27538 rpc_server.cc:307] RPC server started. Bound to: 127.12.138.194:37867
I20260811 02:54:32.955968 27538 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786416707705494-12843-0/minicluster-data/ts-1/data/info.pb
I20260811 02:54:32.957659 12843 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDawEwS/build/debug/bin/kudu as pid 27538
I20260811 02:54:32.957803 12843 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786416707705494-12843-0/minicluster-data/ts-1/wal/instance
I20260811 02:54:32.958159 12843 external_mini_cluster.cc:1428] Setting key 672f93944a49527c6e7253056a7a3b9e
I20260811 02:54:32.957463 27682 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.138.194:37867 every 8 connection(s)
WARNING: no policy specified for kudu/127.12.138.195@KRBTEST.COM; defaulting to no policy
Aug 11 02:54:32 dist-test-slave-9gcw krb5kdc[26726](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416872, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.138.194@KRBTEST.COM for kudu/127.12.138.254@KRBTEST.COM
I20260811 02:54:32.998495 27533 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:54:32.965919 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.138.194:59457 (local address 127.12.138.254:35723)
0811 02:54:32.966049 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0811 02:54:32.966054 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 02:54:32.973813 (+ 7759us) server_negotiation.cc:407] Connection header received
0811 02:54:32.977313 (+ 3500us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:54:32.977318 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:54:32.977384 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:54:32.977441 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:54:32.979248 (+ 1807us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:54:32.980210 (+ 962us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:54:32.984790 (+ 4580us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:54:32.985188 (+ 398us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:54:32.985275 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:54:32.991917 (+ 6642us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:54:32.991922 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:54:32.991925 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:54:32.991956 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:54:32.994061 (+ 2105us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:54:32.995048 (+ 987us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:54:32.995052 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:54:32.995054 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:54:32.995164 (+ 110us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:54:32.996737 (+ 1573us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:54:32.996741 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:54:32.996743 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:54:32.997023 (+ 280us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:54:32.997132 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0811 02:54:32.998125 (+ 993us) server_negotiation.cc:299] Negotiation successful
0811 02:54:32.998301 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.138.195@KRBTEST.COM" created.
I20260811 02:54:33.003786 27688 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:54:32.964568 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.138.254:35723 (local address 127.12.138.194:59457)
0811 02:54:32.973601 (+ 9033us) negotiation.cc:107] Waiting for socket to connect
0811 02:54:32.973675 (+ 74us) client_negotiation.cc:174] Beginning negotiation
0811 02:54:32.974901 (+ 1226us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:54:32.977935 (+ 3034us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:54:32.977946 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:54:32.978352 (+ 406us) client_negotiation.cc:189] Negotiated authn=SASL
0811 02:54:32.979054 (+ 702us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:54:32.979073 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:54:32.980379 (+ 1306us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:54:32.980382 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:54:32.984624 (+ 4242us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:54:32.984637 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:54:32.985575 (+ 938us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:54:32.985580 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:54:32.985804 (+ 224us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:54:32.989858 (+ 4054us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 02:54:32.989882 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0811 02:54:32.991663 (+ 1781us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 02:54:32.994307 (+ 2644us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:54:32.994315 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:54:32.994336 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:54:32.994892 (+ 556us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:54:32.995306 (+ 414us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:54:32.995309 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:54:32.995311 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:54:32.996108 (+ 797us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:54:32.997238 (+ 1130us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 02:54:32.997245 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 02:54:32.997637 (+ 392us) client_negotiation.cc:769] Sending connection context
0811 02:54:32.999242 (+ 1605us) client_negotiation.cc:240] Negotiation successful
0811 02:54:33.002550 (+ 3308us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":8725,"thread_start_us":180,"threads_started":1}
I20260811 02:54:33.005673 27683 heartbeater.cc:347] Connected to a master server at 127.12.138.254:35723
I20260811 02:54:33.005963 27683 heartbeater.cc:464] Registering TS with master...
I20260811 02:54:33.006642 27683 heartbeater.cc:511] Master 127.12.138.254:35723 requested a full tablet report, sending...
I20260811 02:54:33.008248 27284 ts_manager.cc:194] Registered new tserver with Master: 685778c8f32c4f4eae047d64c7907f89 (127.12.138.194:37867)
I20260811 02:54:33.009105 27284 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.12.138.194@KRBTEST.COM'} at 127.12.138.194:59457
Aug 11 02:54:33 dist-test-slave-9gcw krb5kdc[26726](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416872, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.138.254@KRBTEST.COM for HTTP/127.12.138.212@KRBTEST.COM
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.138.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786416707705494-12843-0/krb5kdc/kudu_127.12.138.195.keytab.
Entry for principal kudu/127.12.138.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786416707705494-12843-0/krb5kdc/kudu_127.12.138.195.keytab.
I20260811 02:54:33.059515 12843 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDawEwS/build/debug/bin/kudu
/tmp/dist-test-taskDawEwS/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786416707705494-12843-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786416707705494-12843-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786416707705494-12843-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786416707705494-12843-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.12.138.212:36171/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.138.195:0
--local_ip_for_outbound_sockets=127.12.138.195
--webserver_interface=127.12.138.195
--webserver_port=0
--tserver_master_addrs=127.12.138.254:35723
--keytab_file=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786416707705494-12843-0/krb5kdc/kudu_127.12.138.195.keytab
--principal=kudu/127.12.138.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.138.212:46223
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786416707705494-12843-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786416707705494-12843-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786416707705494-12843-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 02:54:33.215361 27691 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 02:54:33.215744 27691 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 02:54:33.215812 27691 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 02:54:33.222106 27691 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 02:54:33.222196 27691 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 02:54:33.222308 27691 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.138.195
I20260811 02:54:33.230310 27691 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.138.212:46223
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786416707705494-12843-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786416707705494-12843-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.12.138.212:36171/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786416707705494-12843-0/krb5kdc/kudu_127.12.138.195.keytab
--principal=kudu/127.12.138.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.138.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786416707705494-12843-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.12.138.195
--webserver_port=0
--tserver_master_addrs=127.12.138.254:35723
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27691
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.138.195
--log_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786416707705494-12843-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 02:09:49 UTC on ccycloud.quasar-fsnqqc.root.comops.site
build id 4579570
I20260811 02:54:33.231758 27691 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 02:54:33.233193 27691 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 02:54:33.246012 27696 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:54:33.246297 27691 server_base.cc:1034] running on GCE node
W20260811 02:54:33.246037 27697 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:54:33.246493 27701 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:54:33.247272 27691 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 02:54:33.248131 27691 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 02:54:33.249380 27691 hybrid_clock.cc:648] HybridClock initialized: now 1786416873249360 us; error 33 us; skew 500 ppm
Aug 11 02:54:33 dist-test-slave-9gcw krb5kdc[26726](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416873, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.138.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 02:54:33.253114 27691 init.cc:377] Logged in from keytab as kudu/127.12.138.195@KRBTEST.COM (short username kudu)
I20260811 02:54:33.254366 27691 webserver.cc:460] Webserver started at http://127.12.138.195:40799/ using document root <none> and password file <none>
I20260811 02:54:33.255128 27691 fs_manager.cc:359] Metadata directory not provided
I20260811 02:54:33.255287 27691 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 02:54:33.255587 27691 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 11 02:54:33 dist-test-slave-9gcw krb5kdc[26726](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416873, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.138.195@KRBTEST.COM for HTTP/127.12.138.212@KRBTEST.COM
I20260811 02:54:33.272583 27691 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786416707705494-12843-0/minicluster-data/ts-2/data/instance:
uuid: "2c4dc57ad86645a7b1215ae2baad988b"
format_stamp: "Formatted at 2026-08-11 02:54:33 on dist-test-slave-9gcw"
server_key: "69636a3001d07a630844911a359cd428"
server_key_iv: "178faecab07df5adc84398904864f8d8"
server_key_version: "kuduclusterkey@0"
I20260811 02:54:33.273761 27691 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786416707705494-12843-0/minicluster-data/ts-2/wal/instance:
uuid: "2c4dc57ad86645a7b1215ae2baad988b"
format_stamp: "Formatted at 2026-08-11 02:54:33 on dist-test-slave-9gcw"
server_key: "69636a3001d07a630844911a359cd428"
server_key_iv: "178faecab07df5adc84398904864f8d8"
server_key_version: "kuduclusterkey@0"
I20260811 02:54:33.280578 27691 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.000s sys 0.004s
I20260811 02:54:33.310451 27709 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 02:54:33.312531 27691 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.001s
I20260811 02:54:33.312692 27691 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786416707705494-12843-0/minicluster-data/ts-2/data,/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786416707705494-12843-0/minicluster-data/ts-2/wal
uuid: "2c4dc57ad86645a7b1215ae2baad988b"
format_stamp: "Formatted at 2026-08-11 02:54:33 on dist-test-slave-9gcw"
server_key: "69636a3001d07a630844911a359cd428"
server_key_iv: "178faecab07df5adc84398904864f8d8"
server_key_version: "kuduclusterkey@0"
I20260811 02:54:33.312853 27691 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786416707705494-12843-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786416707705494-12843-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786416707705494-12843-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 02:54:33.335770 27691 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 02:54:33.339828 27691 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 02:54:33.340111 27691 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 02:54:33.340957 27691 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 02:54:33.342165 27691 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 02:54:33.342237 27691 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 02:54:33.342303 27691 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 02:54:33.342334 27691 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 02:54:33.353896 27691 rpc_server.cc:307] RPC server started. Bound to: 127.12.138.195:34181
I20260811 02:54:33.355556 27691 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786416707705494-12843-0/minicluster-data/ts-2/data/info.pb
I20260811 02:54:33.363081 12843 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDawEwS/build/debug/bin/kudu as pid 27691
I20260811 02:54:33.363217 12843 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786416707705494-12843-0/minicluster-data/ts-2/wal/instance
I20260811 02:54:33.363610 12843 external_mini_cluster.cc:1428] Setting key 4349401a2bfa5049226ebb301fb6fe02
Aug 11 02:54:33 dist-test-slave-9gcw krb5kdc[26726](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416872, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.138.254@KRBTEST.COM for HTTP/127.12.138.212@KRBTEST.COM
I20260811 02:54:33.362599 27837 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.138.195:34181 every 8 connection(s)
Aug 11 02:54:33 dist-test-slave-9gcw krb5kdc[26726](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416873, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.138.195@KRBTEST.COM for kudu/127.12.138.254@KRBTEST.COM
I20260811 02:54:33.383701 27533 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:54:33.364917 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.138.195:54121 (local address 127.12.138.254:35723)
0811 02:54:33.365213 (+ 296us) server_negotiation.cc:206] Beginning negotiation
0811 02:54:33.365218 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 02:54:33.369813 (+ 4595us) server_negotiation.cc:407] Connection header received
0811 02:54:33.371084 (+ 1271us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:54:33.371106 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:54:33.371165 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:54:33.371224 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:54:33.372965 (+ 1741us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:54:33.373790 (+ 825us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:54:33.375661 (+ 1871us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:54:33.375983 (+ 322us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:54:33.376065 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:54:33.379221 (+ 3156us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:54:33.379225 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:54:33.379228 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:54:33.379257 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:54:33.381129 (+ 1872us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:54:33.381820 (+ 691us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:54:33.381824 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:54:33.381825 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:54:33.381918 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:54:33.382354 (+ 436us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:54:33.382357 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:54:33.382359 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:54:33.382587 (+ 228us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:54:33.382659 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0811 02:54:33.383400 (+ 741us) server_negotiation.cc:299] Negotiation successful
0811 02:54:33.383541 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":185}
I20260811 02:54:33.385344 27842 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:54:33.363546 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.138.254:35723 (local address 127.12.138.195:54121)
0811 02:54:33.369631 (+ 6085us) negotiation.cc:107] Waiting for socket to connect
0811 02:54:33.369695 (+ 64us) client_negotiation.cc:174] Beginning negotiation
0811 02:54:33.370820 (+ 1125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:54:33.371681 (+ 861us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:54:33.371690 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:54:33.372060 (+ 370us) client_negotiation.cc:189] Negotiated authn=SASL
0811 02:54:33.372758 (+ 698us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:54:33.372794 (+ 36us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:54:33.374124 (+ 1330us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:54:33.374128 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:54:33.375502 (+ 1374us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:54:33.375514 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:54:33.376276 (+ 762us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:54:33.376279 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:54:33.376482 (+ 203us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:54:33.377225 (+ 743us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 02:54:33.377248 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0811 02:54:33.379019 (+ 1771us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 02:54:33.381307 (+ 2288us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:54:33.381314 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:54:33.381332 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:54:33.381694 (+ 362us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:54:33.382033 (+ 339us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:54:33.382036 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:54:33.382039 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:54:33.382220 (+ 181us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:54:33.382743 (+ 523us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 02:54:33.382750 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 02:54:33.383086 (+ 336us) client_negotiation.cc:769] Sending connection context
0811 02:54:33.383896 (+ 810us) client_negotiation.cc:240] Negotiation successful
0811 02:54:33.384253 (+ 357us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":843,"thread_start_us":150,"threads_started":1}
I20260811 02:54:33.386746 27840 heartbeater.cc:347] Connected to a master server at 127.12.138.254:35723
I20260811 02:54:33.387028 27840 heartbeater.cc:464] Registering TS with master...
I20260811 02:54:33.387674 27840 heartbeater.cc:511] Master 127.12.138.254:35723 requested a full tablet report, sending...
I20260811 02:54:33.388859 27284 ts_manager.cc:194] Registered new tserver with Master: 2c4dc57ad86645a7b1215ae2baad988b (127.12.138.195:34181)
I20260811 02:54:33.389639 27284 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.12.138.195@KRBTEST.COM'} at 127.12.138.195:54121
I20260811 02:54:33.391057 12843 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 02:54:33.391865 12843 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDawEwS/build/debug/bin/kudu with pid 27377
I20260811 02:54:33.417614 12843 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDawEwS/build/debug/bin/kudu with pid 27538
I20260811 02:54:33.427515 12843 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDawEwS/build/debug/bin/kudu with pid 27691
I20260811 02:54:33.441509 12843 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDawEwS/build/debug/bin/kudu with pid 27249
I20260811 02:54:33.454452 12843 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDawEwS/build/debug/bin/kudu
/tmp/dist-test-taskDawEwS/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786416707705494-12843-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786416707705494-12843-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786416707705494-12843-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786416707705494-12843-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.138.254:35723
--webserver_interface=127.12.138.254
--webserver_port=38885
--keytab_file=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786416707705494-12843-0/krb5kdc/kudu_127.12.138.254.keytab
--principal=kudu/127.12.138.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.138.212:46223
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.12.138.254:35723
--ranger_config_path=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786416707705494-12843-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.12.138.212:36171/kms
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.12.138.212:36171/kms with env {KRB5CCNAME=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786416707705494-12843-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786416707705494-12843-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786416707705494-12843-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 02:54:33.570194 27845 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 02:54:33.570458 27845 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 02:54:33.570497 27845 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 02:54:33.574518 27845 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 02:54:33.574579 27845 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 02:54:33.574599 27845 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 02:54:33.574615 27845 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 02:54:33.574630 27845 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 02:54:33.580050 27845 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.138.212:46223
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786416707705494-12843-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786416707705494-12843-0/minicluster-data/master-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.12.138.212:36171/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.12.138.254:35723
--ranger_config_path=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786416707705494-12843-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786416707705494-12843-0/krb5kdc/kudu_127.12.138.254.keytab
--principal=kudu/127.12.138.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.138.254:35723
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786416707705494-12843-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.12.138.254
--webserver_port=38885
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27845
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786416707705494-12843-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 02:09:49 UTC on ccycloud.quasar-fsnqqc.root.comops.site
build id 4579570
I20260811 02:54:33.581071 27845 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 02:54:33.582103 27845 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 02:54:33.590803 27853 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:54:33.590837 27850 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:54:33.590827 27851 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:54:33.591024 27845 server_base.cc:1034] running on GCE node
I20260811 02:54:33.591902 27845 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 02:54:33.593019 27845 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 02:54:33.594216 27845 hybrid_clock.cc:648] HybridClock initialized: now 1786416873594185 us; error 50 us; skew 500 ppm
Aug 11 02:54:33 dist-test-slave-9gcw krb5kdc[26726](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416873, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.138.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 02:54:33.597028 27845 init.cc:377] Logged in from keytab as kudu/127.12.138.254@KRBTEST.COM (short username kudu)
I20260811 02:54:33.597996 27845 webserver.cc:460] Webserver started at http://127.12.138.254:38885/ using document root <none> and password file <none>
I20260811 02:54:33.598524 27845 fs_manager.cc:359] Metadata directory not provided
I20260811 02:54:33.598593 27845 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 11 02:54:33 dist-test-slave-9gcw krb5kdc[26726](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416873, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.138.254@KRBTEST.COM for HTTP/127.12.138.212@KRBTEST.COM
I20260811 02:54:33.673990 27845 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.000s
I20260811 02:54:33.676481 27865 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 02:54:33.677987 27845 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260811 02:54:33.678102 27845 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786416707705494-12843-0/minicluster-data/master-0/data,/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786416707705494-12843-0/minicluster-data/master-0/wal
uuid: "f36ef9766adb417d848c4dcad0b4baf5"
format_stamp: "Formatted at 2026-08-11 02:54:31 on dist-test-slave-9gcw"
server_key: "a425925ced721729633d087deb9eb9ac"
server_key_iv: "08ae368e2c97f1ce68d9f758bc850ffb"
server_key_version: "kuduclusterkey@0"
I20260811 02:54:33.678673 27845 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786416707705494-12843-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786416707705494-12843-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786416707705494-12843-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 02:54:33.691941 27845 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 02:54:33.695082 27845 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 02:54:33.695309 27845 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 02:54:33.702695 27845 rpc_server.cc:307] RPC server started. Bound to: 127.12.138.254:35723
I20260811 02:54:33.702696 27923 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.138.254:35723 every 8 connection(s)
I20260811 02:54:33.703853 27845 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786416707705494-12843-0/minicluster-data/master-0/data/info.pb
I20260811 02:54:33.708161 27925 sys_catalog.cc:263] Verifying existing consensus state
I20260811 02:54:33.709928 27925 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f36ef9766adb417d848c4dcad0b4baf5: Bootstrap starting.
I20260811 02:54:33.713454 12843 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDawEwS/build/debug/bin/kudu as pid 27845
I20260811 02:54:33.713934 12843 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDawEwS/build/debug/bin/kudu
/tmp/dist-test-taskDawEwS/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786416707705494-12843-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786416707705494-12843-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786416707705494-12843-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786416707705494-12843-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.12.138.212:36171/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.138.193:46259
--local_ip_for_outbound_sockets=127.12.138.193
--tserver_master_addrs=127.12.138.254:35723
--webserver_port=38755
--webserver_interface=127.12.138.193
--keytab_file=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786416707705494-12843-0/krb5kdc/kudu_127.12.138.193.keytab
--principal=kudu/127.12.138.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.138.212:46223
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.12.138.212:36171/kms with env {KRB5CCNAME=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786416707705494-12843-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786416707705494-12843-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786416707705494-12843-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 02:54:33.727249 27925 log.cc:824] T 00000000000000000000000000000000 P f36ef9766adb417d848c4dcad0b4baf5: Log is configured to *not* fsync() on all Append() calls
I20260811 02:54:33.734166 27925 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f36ef9766adb417d848c4dcad0b4baf5: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 02:54:33.734615 27925 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f36ef9766adb417d848c4dcad0b4baf5: Bootstrap complete.
I20260811 02:54:33.739043 27925 raft_consensus.cc:348] T 00000000000000000000000000000000 P f36ef9766adb417d848c4dcad0b4baf5 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f36ef9766adb417d848c4dcad0b4baf5" member_type: VOTER last_known_addr { host: "127.12.138.254" port: 35723 } }
I20260811 02:54:33.739516 27925 raft_consensus.cc:729] T 00000000000000000000000000000000 P f36ef9766adb417d848c4dcad0b4baf5 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: f36ef9766adb417d848c4dcad0b4baf5, State: Initialized, Role: FOLLOWER
I20260811 02:54:33.740058 27925 consensus_queue.cc:260] T 00000000000000000000000000000000 P f36ef9766adb417d848c4dcad0b4baf5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f36ef9766adb417d848c4dcad0b4baf5" member_type: VOTER last_known_addr { host: "127.12.138.254" port: 35723 } }
I20260811 02:54:33.740208 27925 raft_consensus.cc:388] T 00000000000000000000000000000000 P f36ef9766adb417d848c4dcad0b4baf5 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 02:54:33.740350 27925 raft_consensus.cc:482] T 00000000000000000000000000000000 P f36ef9766adb417d848c4dcad0b4baf5 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 02:54:33.740471 27925 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f36ef9766adb417d848c4dcad0b4baf5 [term 1 FOLLOWER]: Advancing to term 2
I20260811 02:54:33.742482 27925 raft_consensus.cc:504] T 00000000000000000000000000000000 P f36ef9766adb417d848c4dcad0b4baf5 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f36ef9766adb417d848c4dcad0b4baf5" member_type: VOTER last_known_addr { host: "127.12.138.254" port: 35723 } }
I20260811 02:54:33.742815 27925 leader_election.cc:302] T 00000000000000000000000000000000 P f36ef9766adb417d848c4dcad0b4baf5 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f36ef9766adb417d848c4dcad0b4baf5; no voters:
I20260811 02:54:33.743163 27925 leader_election.cc:288] T 00000000000000000000000000000000 P f36ef9766adb417d848c4dcad0b4baf5 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260811 02:54:33.743251 27929 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f36ef9766adb417d848c4dcad0b4baf5 [term 2 FOLLOWER]: Leader election won for term 2
I20260811 02:54:33.743456 27929 raft_consensus.cc:686] T 00000000000000000000000000000000 P f36ef9766adb417d848c4dcad0b4baf5 [term 2 LEADER]: Becoming Leader. State: Replica: f36ef9766adb417d848c4dcad0b4baf5, State: Running, Role: LEADER
I20260811 02:54:33.743739 27929 consensus_queue.cc:237] T 00000000000000000000000000000000 P f36ef9766adb417d848c4dcad0b4baf5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f36ef9766adb417d848c4dcad0b4baf5" member_type: VOTER last_known_addr { host: "127.12.138.254" port: 35723 } }
I20260811 02:54:33.744439 27925 sys_catalog.cc:564] T 00000000000000000000000000000000 P f36ef9766adb417d848c4dcad0b4baf5 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 02:54:33.744643 27930 sys_catalog.cc:455] T 00000000000000000000000000000000 P f36ef9766adb417d848c4dcad0b4baf5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "f36ef9766adb417d848c4dcad0b4baf5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f36ef9766adb417d848c4dcad0b4baf5" member_type: VOTER last_known_addr { host: "127.12.138.254" port: 35723 } } }
I20260811 02:54:33.744757 27930 sys_catalog.cc:458] T 00000000000000000000000000000000 P f36ef9766adb417d848c4dcad0b4baf5 [sys.catalog]: This master's current role is: LEADER
I20260811 02:54:33.744988 27931 sys_catalog.cc:455] T 00000000000000000000000000000000 P f36ef9766adb417d848c4dcad0b4baf5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f36ef9766adb417d848c4dcad0b4baf5. Latest consensus state: current_term: 2 leader_uuid: "f36ef9766adb417d848c4dcad0b4baf5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f36ef9766adb417d848c4dcad0b4baf5" member_type: VOTER last_known_addr { host: "127.12.138.254" port: 35723 } } }
I20260811 02:54:33.745086 27931 sys_catalog.cc:458] T 00000000000000000000000000000000 P f36ef9766adb417d848c4dcad0b4baf5 [sys.catalog]: This master's current role is: LEADER
I20260811 02:54:33.745396 27933 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 02:54:33.748637 27933 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 02:54:33.749248 27925 ranger_client.cc:318] Using overwritten properties file: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786416707705494-12843-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260811 02:54:33.749300 27933 catalog_manager.cc:1259] Loaded cluster ID: 22c53b2e41254dc8b7511156533262aa
I20260811 02:54:33.749352 27933 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 02:54:33.751675 27933 catalog_manager.cc:1504] Loading token signing keys...
I20260811 02:54:33.752339 27933 catalog_manager.cc:5879] T 00000000000000000000000000000000 P f36ef9766adb417d848c4dcad0b4baf5: Loaded TSK: 0
I20260811 02:54:33.753618 27933 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260811 02:54:33.860116 27927 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 02:54:33.860479 27927 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 02:54:33.860563 27927 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 02:54:33.866577 27927 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 02:54:33.866663 27927 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 02:54:33.866762 27927 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.138.193
I20260811 02:54:33.874571 27927 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.138.212:46223
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786416707705494-12843-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786416707705494-12843-0/minicluster-data/ts-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.12.138.212:36171/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786416707705494-12843-0/krb5kdc/kudu_127.12.138.193.keytab
--principal=kudu/127.12.138.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.138.193:46259
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786416707705494-12843-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.12.138.193
--webserver_port=38755
--tserver_master_addrs=127.12.138.254:35723
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27927
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.138.193
--log_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786416707705494-12843-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 02:09:49 UTC on ccycloud.quasar-fsnqqc.root.comops.site
build id 4579570
I20260811 02:54:33.875950 27927 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 02:54:33.877285 27927 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 02:54:33.892539 27968 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:54:33.892582 27966 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:54:33.892940 27927 server_base.cc:1034] running on GCE node
W20260811 02:54:33.892845 27965 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:54:33.893720 27927 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 02:54:33.894529 27927 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 02:54:33.895721 27927 hybrid_clock.cc:648] HybridClock initialized: now 1786416873895687 us; error 49 us; skew 500 ppm
Aug 11 02:54:33 dist-test-slave-9gcw krb5kdc[26726](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416873, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.138.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 02:54:33.899631 27927 init.cc:377] Logged in from keytab as kudu/127.12.138.193@KRBTEST.COM (short username kudu)
I20260811 02:54:33.900827 27927 webserver.cc:460] Webserver started at http://127.12.138.193:38755/ using document root <none> and password file <none>
I20260811 02:54:33.901592 27927 fs_manager.cc:359] Metadata directory not provided
I20260811 02:54:33.901705 27927 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 11 02:54:33 dist-test-slave-9gcw krb5kdc[26726](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416873, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.138.193@KRBTEST.COM for HTTP/127.12.138.212@KRBTEST.COM
I20260811 02:54:33.932117 27927 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.004s sys 0.000s
I20260811 02:54:33.935568 27977 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 02:54:33.937332 27927 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.000s
I20260811 02:54:33.937489 27927 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786416707705494-12843-0/minicluster-data/ts-0/data,/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786416707705494-12843-0/minicluster-data/ts-0/wal
uuid: "d77db62fe6364395997bd0e8fd8adebc"
format_stamp: "Formatted at 2026-08-11 02:54:32 on dist-test-slave-9gcw"
server_key: "5376e966589ce934156857ff7b3a3aad"
server_key_iv: "e4f41f7764dba46c08d5839a283e88a3"
server_key_version: "kuduclusterkey@0"
I20260811 02:54:33.938248 27927 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786416707705494-12843-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786416707705494-12843-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786416707705494-12843-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 02:54:33.964663 27927 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 02:54:33.968025 27927 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 02:54:33.968266 27927 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 02:54:33.968959 27927 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 02:54:33.970041 27927 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 02:54:33.970093 27927 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 02:54:33.970160 27927 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 02:54:33.970183 27927 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 02:54:33.982175 27927 rpc_server.cc:307] RPC server started. Bound to: 127.12.138.193:46259
I20260811 02:54:33.982196 28110 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.138.193:46259 every 8 connection(s)
I20260811 02:54:33.983304 27927 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786416707705494-12843-0/minicluster-data/ts-0/data/info.pb
I20260811 02:54:33.986362 12843 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDawEwS/build/debug/bin/kudu as pid 27927
I20260811 02:54:33.986855 12843 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDawEwS/build/debug/bin/kudu
/tmp/dist-test-taskDawEwS/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786416707705494-12843-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786416707705494-12843-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786416707705494-12843-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786416707705494-12843-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.12.138.212:36171/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.138.194:37867
--local_ip_for_outbound_sockets=127.12.138.194
--tserver_master_addrs=127.12.138.254:35723
--webserver_port=37669
--webserver_interface=127.12.138.194
--keytab_file=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786416707705494-12843-0/krb5kdc/kudu_127.12.138.194.keytab
--principal=kudu/127.12.138.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.138.212:46223
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.12.138.212:36171/kms with env {KRB5CCNAME=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786416707705494-12843-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786416707705494-12843-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786416707705494-12843-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 11 02:54:34 dist-test-slave-9gcw krb5kdc[26726](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416873, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.138.193@KRBTEST.COM for kudu/127.12.138.254@KRBTEST.COM
I20260811 02:54:34.018702 28114 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:54:33.985792 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.138.254:35723 (local address 127.12.138.193:33039)
0811 02:54:33.989738 (+ 3946us) negotiation.cc:107] Waiting for socket to connect
0811 02:54:33.989800 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0811 02:54:33.990834 (+ 1034us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:54:33.998202 (+ 7368us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:54:33.998214 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:54:33.998624 (+ 410us) client_negotiation.cc:189] Negotiated authn=SASL
0811 02:54:33.999213 (+ 589us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:54:33.999229 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:54:34.001234 (+ 2005us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:54:34.001240 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:54:34.002147 (+ 907us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:54:34.002157 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:54:34.002952 (+ 795us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:54:34.002956 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:54:34.003149 (+ 193us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:54:34.009654 (+ 6505us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 02:54:34.009676 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0811 02:54:34.011674 (+ 1998us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 02:54:34.014699 (+ 3025us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:54:34.014707 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:54:34.014723 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:54:34.015088 (+ 365us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:54:34.015590 (+ 502us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:54:34.015593 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:54:34.015595 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:54:34.015780 (+ 185us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:54:34.016599 (+ 819us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 02:54:34.016606 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 02:54:34.016928 (+ 322us) client_negotiation.cc:769] Sending connection context
0811 02:54:34.017127 (+ 199us) client_negotiation.cc:240] Negotiation successful
0811 02:54:34.017549 (+ 422us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3700,"thread_start_us":125,"threads_started":1}
I20260811 02:54:34.021479 28113 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:54:33.985635 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.138.193:33039 (local address 127.12.138.254:35723)
0811 02:54:33.989378 (+ 3743us) server_negotiation.cc:206] Beginning negotiation
0811 02:54:33.989406 (+ 28us) server_negotiation.cc:399] Waiting for connection header
0811 02:54:33.989945 (+ 539us) server_negotiation.cc:407] Connection header received
0811 02:54:33.991298 (+ 1353us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:54:33.991310 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:54:33.997576 (+ 6266us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:54:33.999423 (+ 1847us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:54:33.999673 (+ 250us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:54:34.001029 (+ 1356us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:54:34.002282 (+ 1253us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:54:34.002769 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:54:34.002946 (+ 177us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:54:34.011962 (+ 9016us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:54:34.011970 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:54:34.011989 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:54:34.012028 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:54:34.014526 (+ 2498us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:54:34.015274 (+ 748us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:54:34.015281 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:54:34.015288 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:54:34.015464 (+ 176us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:54:34.015907 (+ 443us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:54:34.015910 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:54:34.015913 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:54:34.016455 (+ 542us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:54:34.019949 (+ 3494us) server_negotiation.cc:1035] Waiting for connection context
0811 02:54:34.020117 (+ 168us) server_negotiation.cc:299] Negotiation successful
0811 02:54:34.020463 (+ 346us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3528,"thread_start_us":124,"threads_started":1}
I20260811 02:54:34.024425 28111 heartbeater.cc:347] Connected to a master server at 127.12.138.254:35723
I20260811 02:54:34.024708 28111 heartbeater.cc:464] Registering TS with master...
I20260811 02:54:34.025327 28111 heartbeater.cc:511] Master 127.12.138.254:35723 requested a full tablet report, sending...
W20260811 02:54:34.028221 28111 heartbeater.cc:644] Failed to heartbeat to 127.12.138.254:35723 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260811 02:54:34.028355 28111 heartbeater.cc:464] Registering TS with master...
I20260811 02:54:34.028543 28111 heartbeater.cc:511] Master 127.12.138.254:35723 requested a full tablet report, sending...
I20260811 02:54:34.029629 28111 heartbeater.cc:464] Registering TS with master...
I20260811 02:54:34.029812 28111 heartbeater.cc:511] Master 127.12.138.254:35723 requested a full tablet report, sending...
W20260811 02:54:34.030704 28111 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260811 02:54:34.177991 28115 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 02:54:34.178465 28115 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 02:54:34.178566 28115 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 02:54:34.185674 28115 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 02:54:34.185770 28115 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 02:54:34.185882 28115 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.138.194
I20260811 02:54:34.192770 28115 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.138.212:46223
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786416707705494-12843-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786416707705494-12843-0/minicluster-data/ts-1/wal
--ranger_kms_url=invalid.host:1234/kms,127.12.138.212:36171/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786416707705494-12843-0/krb5kdc/kudu_127.12.138.194.keytab
--principal=kudu/127.12.138.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.138.194:37867
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786416707705494-12843-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.12.138.194
--webserver_port=37669
--tserver_master_addrs=127.12.138.254:35723
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28115
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.138.194
--log_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786416707705494-12843-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 02:09:49 UTC on ccycloud.quasar-fsnqqc.root.comops.site
build id 4579570
I20260811 02:54:34.194223 28115 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 02:54:34.195571 28115 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 02:54:34.206387 28121 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:54:34.206404 28122 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:54:34.206403 28124 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:54:34.207228 28115 server_base.cc:1034] running on GCE node
I20260811 02:54:34.207778 28115 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 02:54:34.208642 28115 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 02:54:34.209856 28115 hybrid_clock.cc:648] HybridClock initialized: now 1786416874209828 us; error 36 us; skew 500 ppm
Aug 11 02:54:34 dist-test-slave-9gcw krb5kdc[26726](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416874, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.138.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 02:54:34.213410 28115 init.cc:377] Logged in from keytab as kudu/127.12.138.194@KRBTEST.COM (short username kudu)
I20260811 02:54:34.214516 28115 webserver.cc:460] Webserver started at http://127.12.138.194:37669/ using document root <none> and password file <none>
I20260811 02:54:34.215185 28115 fs_manager.cc:359] Metadata directory not provided
I20260811 02:54:34.215312 28115 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 11 02:54:34 dist-test-slave-9gcw krb5kdc[26726](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416874, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.138.194@KRBTEST.COM for HTTP/127.12.138.212@KRBTEST.COM
I20260811 02:54:34.246853 28115 fs_manager.cc:709] Time spent opening directory manager: real 0.013s user 0.002s sys 0.001s
I20260811 02:54:34.254666 28132 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 02:54:34.256135 28115 fs_manager.cc:725] Time spent opening block manager: real 0.008s user 0.001s sys 0.000s
I20260811 02:54:34.256289 28115 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786416707705494-12843-0/minicluster-data/ts-1/data,/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786416707705494-12843-0/minicluster-data/ts-1/wal
uuid: "685778c8f32c4f4eae047d64c7907f89"
format_stamp: "Formatted at 2026-08-11 02:54:32 on dist-test-slave-9gcw"
server_key: "4d05b9be606378564458792f405011b4"
server_key_iv: "f14bc8f813b1e8767fbbf6c7c4aa731f"
server_key_version: "kuduclusterkey@0"
I20260811 02:54:34.256919 28115 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786416707705494-12843-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786416707705494-12843-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786416707705494-12843-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 02:54:34.273972 28115 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 02:54:34.277740 28115 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 02:54:34.277976 28115 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 02:54:34.278753 28115 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 02:54:34.279745 28115 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 02:54:34.279822 28115 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 02:54:34.279888 28115 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 02:54:34.279922 28115 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 02:54:34.292440 28115 rpc_server.cc:307] RPC server started. Bound to: 127.12.138.194:37867
I20260811 02:54:34.292460 28263 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.138.194:37867 every 8 connection(s)
I20260811 02:54:34.293725 28115 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786416707705494-12843-0/minicluster-data/ts-1/data/info.pb
I20260811 02:54:34.299319 12843 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDawEwS/build/debug/bin/kudu as pid 28115
I20260811 02:54:34.299865 12843 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDawEwS/build/debug/bin/kudu
/tmp/dist-test-taskDawEwS/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786416707705494-12843-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786416707705494-12843-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786416707705494-12843-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786416707705494-12843-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.12.138.212:36171/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.138.195:34181
--local_ip_for_outbound_sockets=127.12.138.195
--tserver_master_addrs=127.12.138.254:35723
--webserver_port=40799
--webserver_interface=127.12.138.195
--keytab_file=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786416707705494-12843-0/krb5kdc/kudu_127.12.138.195.keytab
--principal=kudu/127.12.138.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.138.212:46223
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.12.138.212:36171/kms with env {KRB5CCNAME=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786416707705494-12843-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786416707705494-12843-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786416707705494-12843-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 11 02:54:34 dist-test-slave-9gcw krb5kdc[26726](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416874, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.138.194@KRBTEST.COM for kudu/127.12.138.254@KRBTEST.COM
I20260811 02:54:34.316486 28113 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:54:34.295821 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.138.194:41559 (local address 127.12.138.254:35723)
0811 02:54:34.296009 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0811 02:54:34.296014 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 02:54:34.296980 (+ 966us) server_negotiation.cc:407] Connection header received
0811 02:54:34.298039 (+ 1059us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:54:34.298042 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:54:34.298098 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:54:34.298184 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:54:34.299796 (+ 1612us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:54:34.300599 (+ 803us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:54:34.302476 (+ 1877us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:54:34.302852 (+ 376us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:54:34.302939 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:54:34.309964 (+ 7025us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:54:34.309971 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:54:34.309976 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:54:34.310019 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:54:34.312145 (+ 2126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:54:34.312994 (+ 849us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:54:34.313001 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:54:34.313004 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:54:34.313110 (+ 106us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:54:34.315015 (+ 1905us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:54:34.315022 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:54:34.315026 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:54:34.315281 (+ 255us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:54:34.315397 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0811 02:54:34.316151 (+ 754us) server_negotiation.cc:299] Negotiation successful
0811 02:54:34.316300 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260811 02:54:34.318869 28267 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:54:34.296314 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.138.254:35723 (local address 127.12.138.194:41559)
0811 02:54:34.296824 (+ 510us) negotiation.cc:107] Waiting for socket to connect
0811 02:54:34.296876 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0811 02:54:34.297817 (+ 941us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:54:34.298516 (+ 699us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:54:34.298525 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:54:34.298884 (+ 359us) client_negotiation.cc:189] Negotiated authn=SASL
0811 02:54:34.299606 (+ 722us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:54:34.299631 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:54:34.300777 (+ 1146us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:54:34.300780 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:54:34.302320 (+ 1540us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:54:34.302336 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:54:34.303167 (+ 831us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:54:34.303172 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:54:34.303425 (+ 253us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:54:34.304134 (+ 709us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 02:54:34.304151 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 02:54:34.309656 (+ 5505us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 02:54:34.312341 (+ 2685us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:54:34.312352 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:54:34.312376 (+ 24us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:54:34.312849 (+ 473us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:54:34.313299 (+ 450us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:54:34.313306 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:54:34.313310 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:54:34.314868 (+ 1558us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:54:34.315470 (+ 602us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 02:54:34.315477 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 02:54:34.315854 (+ 377us) client_negotiation.cc:769] Sending connection context
0811 02:54:34.317251 (+ 1397us) client_negotiation.cc:240] Negotiation successful
0811 02:54:34.317638 (+ 387us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":292,"thread_start_us":150,"threads_started":1}
I20260811 02:54:34.321894 28264 heartbeater.cc:347] Connected to a master server at 127.12.138.254:35723
I20260811 02:54:34.322152 28264 heartbeater.cc:464] Registering TS with master...
I20260811 02:54:34.322997 28264 heartbeater.cc:511] Master 127.12.138.254:35723 requested a full tablet report, sending...
W20260811 02:54:34.325094 28264 heartbeater.cc:644] Failed to heartbeat to 127.12.138.254:35723 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260811 02:54:34.325277 28264 heartbeater.cc:464] Registering TS with master...
I20260811 02:54:34.325479 28264 heartbeater.cc:511] Master 127.12.138.254:35723 requested a full tablet report, sending...
I20260811 02:54:34.326418 28264 heartbeater.cc:464] Registering TS with master...
I20260811 02:54:34.326642 28264 heartbeater.cc:511] Master 127.12.138.254:35723 requested a full tablet report, sending...
W20260811 02:54:34.327518 28264 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260811 02:54:34.451745 28268 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 02:54:34.452066 28268 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 02:54:34.452148 28268 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 02:54:34.456844 28268 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 02:54:34.456929 28268 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 02:54:34.457018 28268 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.138.195
I20260811 02:54:34.464138 28268 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.138.212:46223
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786416707705494-12843-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786416707705494-12843-0/minicluster-data/ts-2/wal
--ranger_kms_url=invalid.host:1234/kms,127.12.138.212:36171/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786416707705494-12843-0/krb5kdc/kudu_127.12.138.195.keytab
--principal=kudu/127.12.138.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.138.195:34181
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786416707705494-12843-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.12.138.195
--webserver_port=40799
--tserver_master_addrs=127.12.138.254:35723
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28268
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.138.195
--log_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786416707705494-12843-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 02:09:49 UTC on ccycloud.quasar-fsnqqc.root.comops.site
build id 4579570
I20260811 02:54:34.465713 28268 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 02:54:34.467042 28268 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 02:54:34.478327 28275 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:54:34.478420 28274 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:54:34.478605 28277 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:54:34.479326 28268 server_base.cc:1034] running on GCE node
I20260811 02:54:34.479761 28268 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 02:54:34.480499 28268 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 02:54:34.481709 28268 hybrid_clock.cc:648] HybridClock initialized: now 1786416874481656 us; error 47 us; skew 500 ppm
Aug 11 02:54:34 dist-test-slave-9gcw krb5kdc[26726](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416874, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.138.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 02:54:34.484730 28268 init.cc:377] Logged in from keytab as kudu/127.12.138.195@KRBTEST.COM (short username kudu)
I20260811 02:54:34.485931 28268 webserver.cc:460] Webserver started at http://127.12.138.195:40799/ using document root <none> and password file <none>
I20260811 02:54:34.486529 28268 fs_manager.cc:359] Metadata directory not provided
I20260811 02:54:34.486644 28268 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 11 02:54:34 dist-test-slave-9gcw krb5kdc[26726](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416874, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.138.195@KRBTEST.COM for HTTP/127.12.138.212@KRBTEST.COM
I20260811 02:54:34.503906 28268 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.000s
I20260811 02:54:34.507258 28285 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 02:54:34.508915 28268 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260811 02:54:34.509063 28268 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786416707705494-12843-0/minicluster-data/ts-2/data,/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786416707705494-12843-0/minicluster-data/ts-2/wal
uuid: "2c4dc57ad86645a7b1215ae2baad988b"
format_stamp: "Formatted at 2026-08-11 02:54:33 on dist-test-slave-9gcw"
server_key: "69636a3001d07a630844911a359cd428"
server_key_iv: "178faecab07df5adc84398904864f8d8"
server_key_version: "kuduclusterkey@0"
I20260811 02:54:34.509886 28268 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786416707705494-12843-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786416707705494-12843-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786416707705494-12843-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 02:54:34.532272 28268 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 02:54:34.536057 28268 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 02:54:34.536341 28268 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 02:54:34.537180 28268 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 02:54:34.538359 28268 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 02:54:34.538436 28268 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 02:54:34.538501 28268 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 02:54:34.538543 28268 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 02:54:34.553928 28268 rpc_server.cc:307] RPC server started. Bound to: 127.12.138.195:34181
I20260811 02:54:34.555528 28268 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786416707705494-12843-0/minicluster-data/ts-2/data/info.pb
I20260811 02:54:34.558292 28414 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.138.195:34181 every 8 connection(s)
I20260811 02:54:34.561748 12843 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDawEwS/build/debug/bin/kudu as pid 28268
Aug 11 02:54:34 dist-test-slave-9gcw krb5kdc[26726](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416874, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.138.195@KRBTEST.COM for kudu/127.12.138.254@KRBTEST.COM
I20260811 02:54:34.582229 28113 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:54:34.559219 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.138.195:49751 (local address 127.12.138.254:35723)
0811 02:54:34.559372 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0811 02:54:34.559378 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 02:54:34.560496 (+ 1118us) server_negotiation.cc:407] Connection header received
0811 02:54:34.561996 (+ 1500us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:54:34.562001 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:54:34.562067 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:54:34.562159 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:54:34.564373 (+ 2214us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:54:34.565249 (+ 876us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:54:34.566950 (+ 1701us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:54:34.567316 (+ 366us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:54:34.567457 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:54:34.570540 (+ 3083us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:54:34.570545 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:54:34.570548 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:54:34.570580 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:54:34.572408 (+ 1828us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:54:34.577314 (+ 4906us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:54:34.577319 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:54:34.577321 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:54:34.577442 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:54:34.578621 (+ 1179us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:54:34.578626 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:54:34.578629 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:54:34.578900 (+ 271us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:54:34.579004 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0811 02:54:34.581865 (+ 2861us) server_negotiation.cc:299] Negotiation successful
0811 02:54:34.582020 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260811 02:54:34.584697 28417 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:54:34.559701 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.138.254:35723 (local address 127.12.138.195:49751)
0811 02:54:34.560308 (+ 607us) negotiation.cc:107] Waiting for socket to connect
0811 02:54:34.560371 (+ 63us) client_negotiation.cc:174] Beginning negotiation
0811 02:54:34.561697 (+ 1326us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:54:34.562993 (+ 1296us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:54:34.563003 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:54:34.563401 (+ 398us) client_negotiation.cc:189] Negotiated authn=SASL
0811 02:54:34.564155 (+ 754us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:54:34.564181 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:54:34.565393 (+ 1212us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:54:34.565397 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:54:34.566809 (+ 1412us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:54:34.566820 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:54:34.567511 (+ 691us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:54:34.567516 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:54:34.567774 (+ 258us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:54:34.568641 (+ 867us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 02:54:34.568663 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0811 02:54:34.570339 (+ 1676us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 02:54:34.575900 (+ 5561us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:54:34.575908 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:54:34.575928 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:54:34.576330 (+ 402us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:54:34.577598 (+ 1268us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:54:34.577604 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:54:34.577609 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:54:34.577851 (+ 242us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:54:34.579103 (+ 1252us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 02:54:34.579111 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 02:54:34.579504 (+ 393us) client_negotiation.cc:769] Sending connection context
0811 02:54:34.582897 (+ 3393us) client_negotiation.cc:240] Negotiation successful
0811 02:54:34.583333 (+ 436us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":351,"thread_start_us":131,"threads_started":1}
I20260811 02:54:34.586150 28415 heartbeater.cc:347] Connected to a master server at 127.12.138.254:35723
I20260811 02:54:34.586485 28415 heartbeater.cc:464] Registering TS with master...
I20260811 02:54:34.587118 28415 heartbeater.cc:511] Master 127.12.138.254:35723 requested a full tablet report, sending...
W20260811 02:54:34.588847 28415 heartbeater.cc:644] Failed to heartbeat to 127.12.138.254:35723 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260811 02:54:34.588989 28415 heartbeater.cc:464] Registering TS with master...
I20260811 02:54:34.589221 28415 heartbeater.cc:511] Master 127.12.138.254:35723 requested a full tablet report, sending...
I20260811 02:54:34.589550 28418 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:54:34.562519 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37720 (local address 127.12.138.254:35723)
0811 02:54:34.565418 (+ 2899us) server_negotiation.cc:206] Beginning negotiation
0811 02:54:34.565423 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 02:54:34.565436 (+ 13us) server_negotiation.cc:407] Connection header received
0811 02:54:34.565472 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:54:34.565474 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:54:34.565519 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:54:34.565610 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:54:34.567814 (+ 2204us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:54:34.568718 (+ 904us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:54:34.571149 (+ 2431us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:54:34.571519 (+ 370us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:54:34.571651 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:54:34.581325 (+ 9674us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:54:34.581330 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:54:34.581333 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:54:34.581363 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:54:34.585908 (+ 4545us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:54:34.586503 (+ 595us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:54:34.586507 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:54:34.586509 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:54:34.586608 (+ 99us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:54:34.588362 (+ 1754us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:54:34.588366 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:54:34.588369 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:54:34.588623 (+ 254us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:54:34.588855 (+ 232us) server_negotiation.cc:1035] Waiting for connection context
0811 02:54:34.589209 (+ 354us) server_negotiation.cc:299] Negotiation successful
0811 02:54:34.589354 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2817,"thread_start_us":116,"threads_started":1}
I20260811 02:54:34.590198 28415 heartbeater.cc:464] Registering TS with master...
I20260811 02:54:34.590559 28415 heartbeater.cc:511] Master 127.12.138.254:35723 requested a full tablet report, sending...
W20260811 02:54:34.591534 28415 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
Aug 11 02:54:34 dist-test-slave-9gcw krb5kdc[26726](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786416874, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.138.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 02:54:35.030961 28111 heartbeater.cc:464] Registering TS with master...
I20260811 02:54:35.031292 28111 heartbeater.cc:511] Master 127.12.138.254:35723 requested a full tablet report, sending...
I20260811 02:54:35.032500 27884 ts_manager.cc:194] Registered new tserver with Master: d77db62fe6364395997bd0e8fd8adebc (127.12.138.193:46259)
I20260811 02:54:35.033846 27884 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.12.138.193@KRBTEST.COM'} at 127.12.138.193:33039
I20260811 02:54:35.327772 28264 heartbeater.cc:464] Registering TS with master...
I20260811 02:54:35.328116 28264 heartbeater.cc:511] Master 127.12.138.254:35723 requested a full tablet report, sending...
I20260811 02:54:35.328897 27884 ts_manager.cc:194] Registered new tserver with Master: 685778c8f32c4f4eae047d64c7907f89 (127.12.138.194:37867)
I20260811 02:54:35.329865 27884 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.12.138.194@KRBTEST.COM'} at 127.12.138.194:41559
Aug 11 02:54:35 dist-test-slave-9gcw krb5kdc[26726](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416874, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.138.254@KRBTEST.COM for HTTP/127.12.138.212@KRBTEST.COM
I20260811 02:54:35.591809 28415 heartbeater.cc:464] Registering TS with master...
I20260811 02:54:35.592132 28415 heartbeater.cc:511] Master 127.12.138.254:35723 requested a full tablet report, sending...
I20260811 02:54:35.593001 27884 ts_manager.cc:194] Registered new tserver with Master: 2c4dc57ad86645a7b1215ae2baad988b (127.12.138.195:34181)
I20260811 02:54:35.593842 27884 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.12.138.195@KRBTEST.COM'} at 127.12.138.195:49751
I20260811 02:54:35.598102 12843 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 02:54:35.609578 28443 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:54:35.599957 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37736 (local address 127.12.138.254:35723)
0811 02:54:35.600246 (+ 289us) server_negotiation.cc:206] Beginning negotiation
0811 02:54:35.600252 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 02:54:35.600269 (+ 17us) server_negotiation.cc:407] Connection header received
0811 02:54:35.600403 (+ 134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:54:35.600409 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:54:35.600464 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:54:35.600559 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:54:35.601427 (+ 868us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:54:35.602289 (+ 862us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:54:35.603411 (+ 1122us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:54:35.603767 (+ 356us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:54:35.603897 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:54:35.605559 (+ 1662us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:54:35.605563 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:54:35.605566 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:54:35.605596 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:54:35.607514 (+ 1918us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:54:35.607997 (+ 483us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:54:35.608001 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:54:35.608003 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:54:35.608104 (+ 101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:54:35.608446 (+ 342us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:54:35.608449 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:54:35.608451 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:54:35.608661 (+ 210us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:54:35.608802 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0811 02:54:35.609091 (+ 289us) server_negotiation.cc:299] Negotiation successful
0811 02:54:35.609247 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":216,"thread_start_us":113,"threads_started":1}
I20260811 02:54:35.614138 27884 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37736:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 02:54:35.619510 27884 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 02:54:35.651507 28449 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:54:35.633600 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.138.195:34181 (local address 127.0.0.1:33828)
0811 02:54:35.642698 (+ 9098us) negotiation.cc:107] Waiting for socket to connect
0811 02:54:35.642713 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 02:54:35.642842 (+ 129us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:54:35.646634 (+ 3792us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:54:35.646638 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:54:35.646658 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 02:54:35.646748 (+ 90us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:54:35.646754 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:54:35.648204 (+ 1450us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:54:35.648208 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:54:35.649688 (+ 1480us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:54:35.649700 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:54:35.650973 (+ 1273us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:54:35.650977 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:54:35.651092 (+ 115us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:54:35.651130 (+ 38us) client_negotiation.cc:769] Sending connection context
0811 02:54:35.651233 (+ 103us) client_negotiation.cc:240] Negotiation successful
0811 02:54:35.651350 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":9026,"mutex_wait_us":111,"thread_start_us":99,"threads_started":1}
I20260811 02:54:35.652122 28455 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:54:35.633932 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33828 (local address 127.12.138.195:34181)
0811 02:54:35.637056 (+ 3124us) server_negotiation.cc:206] Beginning negotiation
0811 02:54:35.637064 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 02:54:35.642817 (+ 5753us) server_negotiation.cc:407] Connection header received
0811 02:54:35.642962 (+ 145us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:54:35.642968 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:54:35.643203 (+ 235us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:54:35.643376 (+ 173us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 02:54:35.646901 (+ 3525us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:54:35.648058 (+ 1157us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:54:35.649841 (+ 1783us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:54:35.650632 (+ 791us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:54:35.650767 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:54:35.650911 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0811 02:54:35.651887 (+ 976us) server_negotiation.cc:299] Negotiation successful
0811 02:54:35.651994 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3002,"thread_start_us":87,"threads_started":1}
I20260811 02:54:35.652420 28452 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:54:35.633424 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43576 (local address 127.12.138.193:46259)
0811 02:54:35.642119 (+ 8695us) server_negotiation.cc:206] Beginning negotiation
0811 02:54:35.642126 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 02:54:35.642150 (+ 24us) server_negotiation.cc:407] Connection header received
0811 02:54:35.642228 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:54:35.642233 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:54:35.642433 (+ 200us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:54:35.645212 (+ 2779us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 02:54:35.645368 (+ 156us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:54:35.646510 (+ 1142us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:54:35.650655 (+ 4145us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:54:35.651453 (+ 798us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:54:35.652028 (+ 575us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:54:35.652136 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0811 02:54:35.652189 (+ 53us) server_negotiation.cc:299] Negotiation successful
0811 02:54:35.652276 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":8574,"thread_start_us":122,"threads_started":1}
I20260811 02:54:35.652422 28456 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:54:35.634000 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44504 (local address 127.12.138.194:37867)
0811 02:54:35.640993 (+ 6993us) server_negotiation.cc:206] Beginning negotiation
0811 02:54:35.641305 (+ 312us) server_negotiation.cc:399] Waiting for connection header
0811 02:54:35.641349 (+ 44us) server_negotiation.cc:407] Connection header received
0811 02:54:35.641458 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:54:35.641465 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:54:35.641781 (+ 316us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:54:35.643377 (+ 1596us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 02:54:35.643536 (+ 159us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:54:35.644707 (+ 1171us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:54:35.649550 (+ 4843us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:54:35.651253 (+ 1703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:54:35.651992 (+ 739us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:54:35.652134 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0811 02:54:35.652189 (+ 55us) server_negotiation.cc:299] Negotiation successful
0811 02:54:35.652277 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":6862,"thread_start_us":111,"threads_started":1}
I20260811 02:54:35.651844 28453 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:54:35.632353 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.138.194:37867 (local address 127.0.0.1:44504)
0811 02:54:35.635320 (+ 2967us) negotiation.cc:107] Waiting for socket to connect
0811 02:54:35.635393 (+ 73us) client_negotiation.cc:174] Beginning negotiation
0811 02:54:35.635526 (+ 133us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:54:35.643008 (+ 7482us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:54:35.643015 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:54:35.643038 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 02:54:35.643216 (+ 178us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:54:35.643223 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:54:35.648257 (+ 5034us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:54:35.648264 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:54:35.649414 (+ 1150us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:54:35.649423 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:54:35.651421 (+ 1998us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:54:35.651425 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:54:35.651527 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:54:35.651562 (+ 35us) client_negotiation.cc:769] Sending connection context
0811 02:54:35.651610 (+ 48us) client_negotiation.cc:240] Negotiation successful
0811 02:54:35.651678 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2769,"mutex_wait_us":49,"thread_start_us":129,"threads_started":1}
I20260811 02:54:35.651948 28451 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:54:35.633257 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.138.193:46259 (local address 127.0.0.1:43576)
0811 02:54:35.635768 (+ 2511us) negotiation.cc:107] Waiting for socket to connect
0811 02:54:35.635780 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 02:54:35.635854 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:54:35.644889 (+ 9035us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:54:35.644895 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:54:35.644919 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 02:54:35.645023 (+ 104us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:54:35.645030 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:54:35.649418 (+ 4388us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:54:35.649421 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:54:35.650494 (+ 1073us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:54:35.650506 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:54:35.651577 (+ 1071us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:54:35.651581 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:54:35.651663 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:54:35.651681 (+ 18us) client_negotiation.cc:769] Sending connection context
0811 02:54:35.651723 (+ 42us) client_negotiation.cc:240] Negotiation successful
0811 02:54:35.651778 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2447,"mutex_wait_us":23,"thread_start_us":109,"threads_started":1}
I20260811 02:54:35.655979 28333 tablet_service.cc:1467] Processing CreateTablet for tablet fbe7db1754bf4585a499a1a985ab93fb (DEFAULT_TABLE table=test-table [id=f107a47a013f40ec9041065d13ea0435]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 02:54:35.655965 28023 tablet_service.cc:1467] Processing CreateTablet for tablet fbe7db1754bf4585a499a1a985ab93fb (DEFAULT_TABLE table=test-table [id=f107a47a013f40ec9041065d13ea0435]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 02:54:35.655967 28190 tablet_service.cc:1467] Processing CreateTablet for tablet fbe7db1754bf4585a499a1a985ab93fb (DEFAULT_TABLE table=test-table [id=f107a47a013f40ec9041065d13ea0435]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 02:54:35.657136 28023 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fbe7db1754bf4585a499a1a985ab93fb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 02:54:35.657136 28190 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fbe7db1754bf4585a499a1a985ab93fb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 02:54:35.657136 28333 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fbe7db1754bf4585a499a1a985ab93fb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 02:54:35.673230 28458 tablet_bootstrap.cc:492] T fbe7db1754bf4585a499a1a985ab93fb P d77db62fe6364395997bd0e8fd8adebc: Bootstrap starting.
Aug 11 02:54:35 dist-test-slave-9gcw krb5kdc[26726](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416874, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.138.254@KRBTEST.COM for HTTP/127.12.138.212@KRBTEST.COM
I20260811 02:54:35.677942 28459 tablet_bootstrap.cc:492] T fbe7db1754bf4585a499a1a985ab93fb P 2c4dc57ad86645a7b1215ae2baad988b: Bootstrap starting.
I20260811 02:54:35.678682 28460 tablet_bootstrap.cc:492] T fbe7db1754bf4585a499a1a985ab93fb P 685778c8f32c4f4eae047d64c7907f89: Bootstrap starting.
I20260811 02:54:35.681253 28460 tablet_bootstrap.cc:654] T fbe7db1754bf4585a499a1a985ab93fb P 685778c8f32c4f4eae047d64c7907f89: Neither blocks nor log segments found. Creating new log.
I20260811 02:54:35.683696 28459 tablet_bootstrap.cc:654] T fbe7db1754bf4585a499a1a985ab93fb P 2c4dc57ad86645a7b1215ae2baad988b: Neither blocks nor log segments found. Creating new log.
I20260811 02:54:35.684474 28459 log.cc:824] T fbe7db1754bf4585a499a1a985ab93fb P 2c4dc57ad86645a7b1215ae2baad988b: Log is configured to *not* fsync() on all Append() calls
I20260811 02:54:35.687384 28458 tablet_bootstrap.cc:654] T fbe7db1754bf4585a499a1a985ab93fb P d77db62fe6364395997bd0e8fd8adebc: Neither blocks nor log segments found. Creating new log.
I20260811 02:54:35.688355 28458 log.cc:824] T fbe7db1754bf4585a499a1a985ab93fb P d77db62fe6364395997bd0e8fd8adebc: Log is configured to *not* fsync() on all Append() calls
I20260811 02:54:35.690682 28459 tablet_bootstrap.cc:492] T fbe7db1754bf4585a499a1a985ab93fb P 2c4dc57ad86645a7b1215ae2baad988b: No bootstrap required, opened a new log
I20260811 02:54:35.690955 28459 ts_tablet_manager.cc:1397] T fbe7db1754bf4585a499a1a985ab93fb P 2c4dc57ad86645a7b1215ae2baad988b: Time spent bootstrapping tablet: real 0.013s user 0.006s sys 0.000s
I20260811 02:54:35.692516 28458 tablet_bootstrap.cc:492] T fbe7db1754bf4585a499a1a985ab93fb P d77db62fe6364395997bd0e8fd8adebc: No bootstrap required, opened a new log
I20260811 02:54:35.692754 28458 ts_tablet_manager.cc:1397] T fbe7db1754bf4585a499a1a985ab93fb P d77db62fe6364395997bd0e8fd8adebc: Time spent bootstrapping tablet: real 0.020s user 0.006s sys 0.000s
I20260811 02:54:35.695408 28459 raft_consensus.cc:348] T fbe7db1754bf4585a499a1a985ab93fb P 2c4dc57ad86645a7b1215ae2baad988b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "685778c8f32c4f4eae047d64c7907f89" member_type: VOTER last_known_addr { host: "127.12.138.194" port: 37867 } } peers { permanent_uuid: "d77db62fe6364395997bd0e8fd8adebc" member_type: VOTER last_known_addr { host: "127.12.138.193" port: 46259 } } peers { permanent_uuid: "2c4dc57ad86645a7b1215ae2baad988b" member_type: VOTER last_known_addr { host: "127.12.138.195" port: 34181 } }
I20260811 02:54:35.695897 28459 raft_consensus.cc:374] T fbe7db1754bf4585a499a1a985ab93fb P 2c4dc57ad86645a7b1215ae2baad988b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 02:54:35.696012 28459 raft_consensus.cc:729] T fbe7db1754bf4585a499a1a985ab93fb P 2c4dc57ad86645a7b1215ae2baad988b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2c4dc57ad86645a7b1215ae2baad988b, State: Initialized, Role: FOLLOWER
I20260811 02:54:35.696593 28459 consensus_queue.cc:260] T fbe7db1754bf4585a499a1a985ab93fb P 2c4dc57ad86645a7b1215ae2baad988b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "685778c8f32c4f4eae047d64c7907f89" member_type: VOTER last_known_addr { host: "127.12.138.194" port: 37867 } } peers { permanent_uuid: "d77db62fe6364395997bd0e8fd8adebc" member_type: VOTER last_known_addr { host: "127.12.138.193" port: 46259 } } peers { permanent_uuid: "2c4dc57ad86645a7b1215ae2baad988b" member_type: VOTER last_known_addr { host: "127.12.138.195" port: 34181 } }
I20260811 02:54:35.696826 28458 raft_consensus.cc:348] T fbe7db1754bf4585a499a1a985ab93fb P d77db62fe6364395997bd0e8fd8adebc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "685778c8f32c4f4eae047d64c7907f89" member_type: VOTER last_known_addr { host: "127.12.138.194" port: 37867 } } peers { permanent_uuid: "d77db62fe6364395997bd0e8fd8adebc" member_type: VOTER last_known_addr { host: "127.12.138.193" port: 46259 } } peers { permanent_uuid: "2c4dc57ad86645a7b1215ae2baad988b" member_type: VOTER last_known_addr { host: "127.12.138.195" port: 34181 } }
I20260811 02:54:35.697134 28458 raft_consensus.cc:374] T fbe7db1754bf4585a499a1a985ab93fb P d77db62fe6364395997bd0e8fd8adebc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 02:54:35.697254 28458 raft_consensus.cc:729] T fbe7db1754bf4585a499a1a985ab93fb P d77db62fe6364395997bd0e8fd8adebc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d77db62fe6364395997bd0e8fd8adebc, State: Initialized, Role: FOLLOWER
I20260811 02:54:35.697819 28458 consensus_queue.cc:260] T fbe7db1754bf4585a499a1a985ab93fb P d77db62fe6364395997bd0e8fd8adebc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "685778c8f32c4f4eae047d64c7907f89" member_type: VOTER last_known_addr { host: "127.12.138.194" port: 37867 } } peers { permanent_uuid: "d77db62fe6364395997bd0e8fd8adebc" member_type: VOTER last_known_addr { host: "127.12.138.193" port: 46259 } } peers { permanent_uuid: "2c4dc57ad86645a7b1215ae2baad988b" member_type: VOTER last_known_addr { host: "127.12.138.195" port: 34181 } }
I20260811 02:54:35.697938 28459 ts_tablet_manager.cc:1428] T fbe7db1754bf4585a499a1a985ab93fb P 2c4dc57ad86645a7b1215ae2baad988b: Time spent starting tablet: real 0.007s user 0.006s sys 0.000s
I20260811 02:54:35.698392 28415 heartbeater.cc:503] Master 127.12.138.254:35723 was elected leader, sending a full tablet report...
I20260811 02:54:35.698968 28458 ts_tablet_manager.cc:1428] T fbe7db1754bf4585a499a1a985ab93fb P d77db62fe6364395997bd0e8fd8adebc: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260811 02:54:35.699330 28111 heartbeater.cc:503] Master 127.12.138.254:35723 was elected leader, sending a full tablet report...
I20260811 02:54:35.702545 28460 log.cc:824] T fbe7db1754bf4585a499a1a985ab93fb P 685778c8f32c4f4eae047d64c7907f89: Log is configured to *not* fsync() on all Append() calls
I20260811 02:54:35.711651 28460 tablet_bootstrap.cc:492] T fbe7db1754bf4585a499a1a985ab93fb P 685778c8f32c4f4eae047d64c7907f89: No bootstrap required, opened a new log
I20260811 02:54:35.711892 28460 ts_tablet_manager.cc:1397] T fbe7db1754bf4585a499a1a985ab93fb P 685778c8f32c4f4eae047d64c7907f89: Time spent bootstrapping tablet: real 0.033s user 0.000s sys 0.005s
I20260811 02:54:35.716730 28460 raft_consensus.cc:348] T fbe7db1754bf4585a499a1a985ab93fb P 685778c8f32c4f4eae047d64c7907f89 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "685778c8f32c4f4eae047d64c7907f89" member_type: VOTER last_known_addr { host: "127.12.138.194" port: 37867 } } peers { permanent_uuid: "d77db62fe6364395997bd0e8fd8adebc" member_type: VOTER last_known_addr { host: "127.12.138.193" port: 46259 } } peers { permanent_uuid: "2c4dc57ad86645a7b1215ae2baad988b" member_type: VOTER last_known_addr { host: "127.12.138.195" port: 34181 } }
I20260811 02:54:35.717819 28460 raft_consensus.cc:374] T fbe7db1754bf4585a499a1a985ab93fb P 685778c8f32c4f4eae047d64c7907f89 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 02:54:35.717900 28460 raft_consensus.cc:729] T fbe7db1754bf4585a499a1a985ab93fb P 685778c8f32c4f4eae047d64c7907f89 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 685778c8f32c4f4eae047d64c7907f89, State: Initialized, Role: FOLLOWER
I20260811 02:54:35.718432 28460 consensus_queue.cc:260] T fbe7db1754bf4585a499a1a985ab93fb P 685778c8f32c4f4eae047d64c7907f89 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "685778c8f32c4f4eae047d64c7907f89" member_type: VOTER last_known_addr { host: "127.12.138.194" port: 37867 } } peers { permanent_uuid: "d77db62fe6364395997bd0e8fd8adebc" member_type: VOTER last_known_addr { host: "127.12.138.193" port: 46259 } } peers { permanent_uuid: "2c4dc57ad86645a7b1215ae2baad988b" member_type: VOTER last_known_addr { host: "127.12.138.195" port: 34181 } }
I20260811 02:54:35.719657 28460 ts_tablet_manager.cc:1428] T fbe7db1754bf4585a499a1a985ab93fb P 685778c8f32c4f4eae047d64c7907f89: Time spent starting tablet: real 0.008s user 0.005s sys 0.001s
I20260811 02:54:35.720057 28264 heartbeater.cc:503] Master 127.12.138.254:35723 was elected leader, sending a full tablet report...
W20260811 02:54:35.736845 28112 tablet.cc:2369] T fbe7db1754bf4585a499a1a985ab93fb P d77db62fe6364395997bd0e8fd8adebc: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 02:54:35.796427 28266 tablet.cc:2369] T fbe7db1754bf4585a499a1a985ab93fb P 685778c8f32c4f4eae047d64c7907f89: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 02:54:35.809927 28416 tablet.cc:2369] T fbe7db1754bf4585a499a1a985ab93fb P 2c4dc57ad86645a7b1215ae2baad988b: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 02:54:35.878328 28463 raft_consensus.cc:482] T fbe7db1754bf4585a499a1a985ab93fb P 2c4dc57ad86645a7b1215ae2baad988b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 02:54:35.878573 28463 raft_consensus.cc:504] T fbe7db1754bf4585a499a1a985ab93fb P 2c4dc57ad86645a7b1215ae2baad988b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "685778c8f32c4f4eae047d64c7907f89" member_type: VOTER last_known_addr { host: "127.12.138.194" port: 37867 } } peers { permanent_uuid: "d77db62fe6364395997bd0e8fd8adebc" member_type: VOTER last_known_addr { host: "127.12.138.193" port: 46259 } } peers { permanent_uuid: "2c4dc57ad86645a7b1215ae2baad988b" member_type: VOTER last_known_addr { host: "127.12.138.195" port: 34181 } }
I20260811 02:54:35.879865 28463 leader_election.cc:288] T fbe7db1754bf4585a499a1a985ab93fb P 2c4dc57ad86645a7b1215ae2baad988b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 685778c8f32c4f4eae047d64c7907f89 (127.12.138.194:37867), d77db62fe6364395997bd0e8fd8adebc (127.12.138.193:46259)
I20260811 02:54:35.885118 28456 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:54:35.880169 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.138.195:60675 (local address 127.12.138.194:37867)
0811 02:54:35.880300 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0811 02:54:35.880305 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 02:54:35.880563 (+ 258us) server_negotiation.cc:407] Connection header received
0811 02:54:35.880807 (+ 244us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:54:35.880811 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:54:35.880864 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:54:35.880922 (+ 58us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 02:54:35.881436 (+ 514us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:54:35.882212 (+ 776us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:54:35.883803 (+ 1591us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:54:35.884389 (+ 586us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:54:35.884476 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:54:35.884530 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0811 02:54:35.884887 (+ 357us) server_negotiation.cc:299] Negotiation successful
0811 02:54:35.884965 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260811 02:54:35.885632 28467 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:54:35.880054 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.138.194:37867 (local address 127.12.138.195:60675)
0811 02:54:35.880466 (+ 412us) negotiation.cc:107] Waiting for socket to connect
0811 02:54:35.880481 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 02:54:35.880653 (+ 172us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:54:35.881091 (+ 438us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:54:35.881094 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:54:35.881111 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 02:54:35.881298 (+ 187us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:54:35.881306 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:54:35.882381 (+ 1075us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:54:35.882384 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:54:35.883660 (+ 1276us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:54:35.883669 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:54:35.884619 (+ 950us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:54:35.884623 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:54:35.884728 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:54:35.884751 (+ 23us) client_negotiation.cc:769] Sending connection context
0811 02:54:35.885361 (+ 610us) client_negotiation.cc:240] Negotiation successful
0811 02:54:35.885436 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":339,"thread_start_us":122,"threads_started":1}
I20260811 02:54:35.885792 28452 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:54:35.880880 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.138.195:38131 (local address 127.12.138.193:46259)
0811 02:54:35.881038 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0811 02:54:35.881044 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 02:54:35.881306 (+ 262us) server_negotiation.cc:407] Connection header received
0811 02:54:35.881492 (+ 186us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:54:35.881496 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:54:35.881543 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:54:35.881589 (+ 46us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 02:54:35.882019 (+ 430us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:54:35.882803 (+ 784us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:54:35.884256 (+ 1453us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:54:35.884833 (+ 577us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:54:35.884920 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:54:35.884964 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0811 02:54:35.885574 (+ 610us) server_negotiation.cc:299] Negotiation successful
0811 02:54:35.885658 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260811 02:54:35.886307 28468 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:54:35.880490 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.138.193:46259 (local address 127.12.138.195:38131)
0811 02:54:35.881213 (+ 723us) negotiation.cc:107] Waiting for socket to connect
0811 02:54:35.881226 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 02:54:35.881371 (+ 145us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:54:35.881715 (+ 344us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:54:35.881718 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:54:35.881733 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 02:54:35.881894 (+ 161us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:54:35.881901 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:54:35.883019 (+ 1118us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:54:35.883022 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:54:35.884129 (+ 1107us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:54:35.884138 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:54:35.885049 (+ 911us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:54:35.885052 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:54:35.885183 (+ 131us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:54:35.885446 (+ 263us) client_negotiation.cc:769] Sending connection context
0811 02:54:35.886108 (+ 662us) client_negotiation.cc:240] Negotiation successful
0811 02:54:35.886186 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":633,"thread_start_us":85,"threads_started":1}
I20260811 02:54:35.886468 28214 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fbe7db1754bf4585a499a1a985ab93fb" candidate_uuid: "2c4dc57ad86645a7b1215ae2baad988b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "685778c8f32c4f4eae047d64c7907f89" is_pre_election: true
I20260811 02:54:35.886855 28214 raft_consensus.cc:2454] T fbe7db1754bf4585a499a1a985ab93fb P 685778c8f32c4f4eae047d64c7907f89 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c4dc57ad86645a7b1215ae2baad988b in term 0.
I20260811 02:54:35.887187 28041 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fbe7db1754bf4585a499a1a985ab93fb" candidate_uuid: "2c4dc57ad86645a7b1215ae2baad988b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d77db62fe6364395997bd0e8fd8adebc" is_pre_election: true
I20260811 02:54:35.887420 28287 leader_election.cc:302] T fbe7db1754bf4585a499a1a985ab93fb P 2c4dc57ad86645a7b1215ae2baad988b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4dc57ad86645a7b1215ae2baad988b, 685778c8f32c4f4eae047d64c7907f89; no voters:
I20260811 02:54:35.887501 28041 raft_consensus.cc:2454] T fbe7db1754bf4585a499a1a985ab93fb P d77db62fe6364395997bd0e8fd8adebc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c4dc57ad86645a7b1215ae2baad988b in term 0.
I20260811 02:54:35.887770 28463 raft_consensus.cc:2781] T fbe7db1754bf4585a499a1a985ab93fb P 2c4dc57ad86645a7b1215ae2baad988b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 02:54:35.887866 28463 raft_consensus.cc:482] T fbe7db1754bf4585a499a1a985ab93fb P 2c4dc57ad86645a7b1215ae2baad988b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 02:54:35.887959 28463 raft_consensus.cc:3037] T fbe7db1754bf4585a499a1a985ab93fb P 2c4dc57ad86645a7b1215ae2baad988b [term 0 FOLLOWER]: Advancing to term 1
I20260811 02:54:35.889437 28463 raft_consensus.cc:504] T fbe7db1754bf4585a499a1a985ab93fb P 2c4dc57ad86645a7b1215ae2baad988b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "685778c8f32c4f4eae047d64c7907f89" member_type: VOTER last_known_addr { host: "127.12.138.194" port: 37867 } } peers { permanent_uuid: "d77db62fe6364395997bd0e8fd8adebc" member_type: VOTER last_known_addr { host: "127.12.138.193" port: 46259 } } peers { permanent_uuid: "2c4dc57ad86645a7b1215ae2baad988b" member_type: VOTER last_known_addr { host: "127.12.138.195" port: 34181 } }
I20260811 02:54:35.889829 28463 leader_election.cc:288] T fbe7db1754bf4585a499a1a985ab93fb P 2c4dc57ad86645a7b1215ae2baad988b [CANDIDATE]: Term 1 election: Requested vote from peers 685778c8f32c4f4eae047d64c7907f89 (127.12.138.194:37867), d77db62fe6364395997bd0e8fd8adebc (127.12.138.193:46259)
I20260811 02:54:35.890187 28214 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fbe7db1754bf4585a499a1a985ab93fb" candidate_uuid: "2c4dc57ad86645a7b1215ae2baad988b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "685778c8f32c4f4eae047d64c7907f89"
I20260811 02:54:35.890326 28214 raft_consensus.cc:3037] T fbe7db1754bf4585a499a1a985ab93fb P 685778c8f32c4f4eae047d64c7907f89 [term 0 FOLLOWER]: Advancing to term 1
I20260811 02:54:35.890328 28041 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fbe7db1754bf4585a499a1a985ab93fb" candidate_uuid: "2c4dc57ad86645a7b1215ae2baad988b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d77db62fe6364395997bd0e8fd8adebc"
I20260811 02:54:35.890466 28041 raft_consensus.cc:3037] T fbe7db1754bf4585a499a1a985ab93fb P d77db62fe6364395997bd0e8fd8adebc [term 0 FOLLOWER]: Advancing to term 1
I20260811 02:54:35.891665 28214 raft_consensus.cc:2454] T fbe7db1754bf4585a499a1a985ab93fb P 685778c8f32c4f4eae047d64c7907f89 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c4dc57ad86645a7b1215ae2baad988b in term 1.
I20260811 02:54:35.891908 28041 raft_consensus.cc:2454] T fbe7db1754bf4585a499a1a985ab93fb P d77db62fe6364395997bd0e8fd8adebc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c4dc57ad86645a7b1215ae2baad988b in term 1.
I20260811 02:54:35.891976 28287 leader_election.cc:302] T fbe7db1754bf4585a499a1a985ab93fb P 2c4dc57ad86645a7b1215ae2baad988b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4dc57ad86645a7b1215ae2baad988b, 685778c8f32c4f4eae047d64c7907f89; no voters:
I20260811 02:54:35.892139 28463 raft_consensus.cc:2781] T fbe7db1754bf4585a499a1a985ab93fb P 2c4dc57ad86645a7b1215ae2baad988b [term 1 FOLLOWER]: Leader election won for term 1
I20260811 02:54:35.892362 28463 raft_consensus.cc:686] T fbe7db1754bf4585a499a1a985ab93fb P 2c4dc57ad86645a7b1215ae2baad988b [term 1 LEADER]: Becoming Leader. State: Replica: 2c4dc57ad86645a7b1215ae2baad988b, State: Running, Role: LEADER
I20260811 02:54:35.892720 28463 consensus_queue.cc:237] T fbe7db1754bf4585a499a1a985ab93fb P 2c4dc57ad86645a7b1215ae2baad988b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "685778c8f32c4f4eae047d64c7907f89" member_type: VOTER last_known_addr { host: "127.12.138.194" port: 37867 } } peers { permanent_uuid: "d77db62fe6364395997bd0e8fd8adebc" member_type: VOTER last_known_addr { host: "127.12.138.193" port: 46259 } } peers { permanent_uuid: "2c4dc57ad86645a7b1215ae2baad988b" member_type: VOTER last_known_addr { host: "127.12.138.195" port: 34181 } }
I20260811 02:54:35.896605 27884 catalog_manager.cc:5495] T fbe7db1754bf4585a499a1a985ab93fb P 2c4dc57ad86645a7b1215ae2baad988b reported cstate change: term changed from 0 to 1, leader changed from <none> to 2c4dc57ad86645a7b1215ae2baad988b (127.12.138.195). New cstate: current_term: 1 leader_uuid: "2c4dc57ad86645a7b1215ae2baad988b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "685778c8f32c4f4eae047d64c7907f89" member_type: VOTER last_known_addr { host: "127.12.138.194" port: 37867 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d77db62fe6364395997bd0e8fd8adebc" member_type: VOTER last_known_addr { host: "127.12.138.193" port: 46259 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2c4dc57ad86645a7b1215ae2baad988b" member_type: VOTER last_known_addr { host: "127.12.138.195" port: 34181 } health_report { overall_health: HEALTHY } } }
I20260811 02:54:35.941619 28455 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:54:35.936915 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33830 (local address 127.12.138.195:34181)
0811 02:54:35.937064 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0811 02:54:35.937069 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 02:54:35.937088 (+ 19us) server_negotiation.cc:407] Connection header received
0811 02:54:35.937210 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:54:35.937214 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:54:35.937287 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:54:35.937425 (+ 138us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 02:54:35.937814 (+ 389us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:54:35.938696 (+ 882us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:54:35.939613 (+ 917us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:54:35.940015 (+ 402us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:54:35.940150 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:54:35.940507 (+ 357us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 02:54:35.941111 (+ 604us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 02:54:35.941293 (+ 182us) server_negotiation.cc:1035] Waiting for connection context
0811 02:54:35.941363 (+ 70us) server_negotiation.cc:299] Negotiation successful
0811 02:54:35.941432 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260811 02:54:35.948439 28041 raft_consensus.cc:1261] T fbe7db1754bf4585a499a1a985ab93fb P d77db62fe6364395997bd0e8fd8adebc [term 1 FOLLOWER]: Refusing update from remote peer 2c4dc57ad86645a7b1215ae2baad988b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 02:54:35.948439 28214 raft_consensus.cc:1261] T fbe7db1754bf4585a499a1a985ab93fb P 685778c8f32c4f4eae047d64c7907f89 [term 1 FOLLOWER]: Refusing update from remote peer 2c4dc57ad86645a7b1215ae2baad988b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 02:54:35.949374 28469 consensus_queue.cc:1035] T fbe7db1754bf4585a499a1a985ab93fb P 2c4dc57ad86645a7b1215ae2baad988b [LEADER]: Connected to new peer: Peer: permanent_uuid: "d77db62fe6364395997bd0e8fd8adebc" member_type: VOTER last_known_addr { host: "127.12.138.193" port: 46259 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 02:54:35.949661 28463 consensus_queue.cc:1035] T fbe7db1754bf4585a499a1a985ab93fb P 2c4dc57ad86645a7b1215ae2baad988b [LEADER]: Connected to new peer: Peer: permanent_uuid: "685778c8f32c4f4eae047d64c7907f89" member_type: VOTER last_known_addr { host: "127.12.138.194" port: 37867 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 02:54:35.959273 28480 mvcc.cc:205] Tried to move back new op lower bound from 7317163523876462592 to 7317163523659784192. Current Snapshot: MvccSnapshot[applied={T|T < 7317163523876462592 or (T in {7317163523876462592})}]
I20260811 02:54:35.968473 27883 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37736:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 02:54:35.968672 27883 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37736:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 02:54:35.971205 27883 catalog_manager.cc:5782] T 00000000000000000000000000000000 P f36ef9766adb417d848c4dcad0b4baf5: Sending DeleteTablet for 3 replicas of tablet fbe7db1754bf4585a499a1a985ab93fb
I20260811 02:54:35.972083 28190 tablet_service.cc:1514] Processing DeleteTablet for tablet fbe7db1754bf4585a499a1a985ab93fb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 02:54:35 UTC) from {username='kudu', principal='kudu/127.12.138.254@KRBTEST.COM'} at 127.0.0.1:44504
I20260811 02:54:35.972148 28023 tablet_service.cc:1514] Processing DeleteTablet for tablet fbe7db1754bf4585a499a1a985ab93fb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 02:54:35 UTC) from {username='kudu', principal='kudu/127.12.138.254@KRBTEST.COM'} at 127.0.0.1:43576
I20260811 02:54:35.972710 28492 tablet_replica.cc:330] T fbe7db1754bf4585a499a1a985ab93fb P 685778c8f32c4f4eae047d64c7907f89: stopping tablet replica
I20260811 02:54:35.972724 28333 tablet_service.cc:1514] Processing DeleteTablet for tablet fbe7db1754bf4585a499a1a985ab93fb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 02:54:35 UTC) from {username='kudu', principal='kudu/127.12.138.254@KRBTEST.COM'} at 127.0.0.1:33828
I20260811 02:54:35.973073 28492 raft_consensus.cc:2229] T fbe7db1754bf4585a499a1a985ab93fb P 685778c8f32c4f4eae047d64c7907f89 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 02:54:35.973307 28494 tablet_replica.cc:330] T fbe7db1754bf4585a499a1a985ab93fb P 2c4dc57ad86645a7b1215ae2baad988b: stopping tablet replica
I20260811 02:54:35.973489 28492 raft_consensus.cc:2258] T fbe7db1754bf4585a499a1a985ab93fb P 685778c8f32c4f4eae047d64c7907f89 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 02:54:35.973732 28494 raft_consensus.cc:2229] T fbe7db1754bf4585a499a1a985ab93fb P 2c4dc57ad86645a7b1215ae2baad988b [term 1 LEADER]: Raft consensus shutting down.
I20260811 02:54:35.974237 28494 raft_consensus.cc:2258] T fbe7db1754bf4585a499a1a985ab93fb P 2c4dc57ad86645a7b1215ae2baad988b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 02:54:35.974314 12843 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDawEwS/build/debug/bin/kudu with pid 27927
I20260811 02:54:35.975801 28492 ts_tablet_manager.cc:1905] T fbe7db1754bf4585a499a1a985ab93fb P 685778c8f32c4f4eae047d64c7907f89: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 02:54:35.976992 28494 ts_tablet_manager.cc:1905] T fbe7db1754bf4585a499a1a985ab93fb P 2c4dc57ad86645a7b1215ae2baad988b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 02:54:35.978909 28492 ts_tablet_manager.cc:1918] T fbe7db1754bf4585a499a1a985ab93fb P 685778c8f32c4f4eae047d64c7907f89: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 02:54:35.979025 28492 log.cc:1196] T fbe7db1754bf4585a499a1a985ab93fb P 685778c8f32c4f4eae047d64c7907f89: Deleting WAL directory at /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786416707705494-12843-0/minicluster-data/ts-1/wal/wals/fbe7db1754bf4585a499a1a985ab93fb
I20260811 02:54:35.979426 28492 ts_tablet_manager.cc:1939] T fbe7db1754bf4585a499a1a985ab93fb P 685778c8f32c4f4eae047d64c7907f89: Deleting consensus metadata
I20260811 02:54:35.980471 28494 ts_tablet_manager.cc:1918] T fbe7db1754bf4585a499a1a985ab93fb P 2c4dc57ad86645a7b1215ae2baad988b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 02:54:35.980635 28494 log.cc:1196] T fbe7db1754bf4585a499a1a985ab93fb P 2c4dc57ad86645a7b1215ae2baad988b: Deleting WAL directory at /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786416707705494-12843-0/minicluster-data/ts-2/wal/wals/fbe7db1754bf4585a499a1a985ab93fb
I20260811 02:54:35.981037 28494 ts_tablet_manager.cc:1939] T fbe7db1754bf4585a499a1a985ab93fb P 2c4dc57ad86645a7b1215ae2baad988b: Deleting consensus metadata
I20260811 02:54:35.981405 27867 catalog_manager.cc:4841] TS 685778c8f32c4f4eae047d64c7907f89 (127.12.138.194:37867): tablet fbe7db1754bf4585a499a1a985ab93fb (table test-table [id=f107a47a013f40ec9041065d13ea0435]) successfully deleted
I20260811 02:54:35.982182 27867 catalog_manager.cc:4841] TS 2c4dc57ad86645a7b1215ae2baad988b (127.12.138.195:34181): tablet fbe7db1754bf4585a499a1a985ab93fb (table test-table [id=f107a47a013f40ec9041065d13ea0435]) successfully deleted
W20260811 02:54:35.989069 27867 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.12.138.193:46259 (error 108)
I20260811 02:54:35.990082 12843 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDawEwS/build/debug/bin/kudu with pid 28115
I20260811 02:54:35.990193 28449 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:54:35.989895 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.138.193:46259 (local address 127.0.0.1:43584)
0811 02:54:35.990023 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0811 02:54:35.990093 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.138.193:46259: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
W20260811 02:54:35.992693 27867 catalog_manager.cc:4584] TS d77db62fe6364395997bd0e8fd8adebc (127.12.138.193:46259): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet fbe7db1754bf4585a499a1a985ab93fb: Network error: Client connection negotiation failed: client connection to 127.12.138.193:46259: connect: Connection refused (error 111)
I20260811 02:54:35.998946 12843 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDawEwS/build/debug/bin/kudu with pid 28268
I20260811 02:54:36.008376 12843 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDawEwS/build/debug/bin/kudu with pid 27845
I20260811 02:54:36.018941 12843 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260811 02:54:36.364588 12843 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260811 02:54:36.364723 12843 mini_ranger.cc:66] Stopping Ranger...
I20260811 02:54:36.709723 12843 mini_ranger.cc:68] Stopped Ranger
2026-08-11 02:54:36.709 UTC [26764] LOG: received smart shutdown request
2026-08-11 02:54:36.714 UTC [26764] LOG: background worker "logical replication launcher" (PID 26773) exited with exit code 1
2026-08-11 02:54:36.715 UTC [26768] LOG: shutting down
2026-08-11 02:54:36.908 UTC [26764] LOG: database system is shut down
2026-08-11T02:54:36Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers (52822 ms)
[ RUN ] SecurityITest.IPKICACert
Loading random data
Initializing database '/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786416707705494-12843-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 02:54:37 dist-test-slave-9gcw krb5kdc[28504](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 02:54:37 dist-test-slave-9gcw krb5kdc[28504](info): set up 2 sockets
Aug 11 02:54:37 dist-test-slave-9gcw krb5kdc[28504](info): commencing operation
krb5kdc: starting...
W20260811 02:54:38.182334 12843 mini_kdc.cc:121] Time spent starting KDC: real 1.122s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786416707705494-12843-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786416707705494-12843-0/krb5kdc/test-user.keytab.
Aug 11 02:54:38 dist-test-slave-9gcw krb5kdc[28504](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416878, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T02:54:38Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T02:54:38Z Disabled control of system clock
WARNING: no policy specified for kudu/127.12.138.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.138.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.138.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786416707705494-12843-0/krb5kdc/kudu_127.12.138.254.keytab.
Entry for principal kudu/127.12.138.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786416707705494-12843-0/krb5kdc/kudu_127.12.138.254.keytab.
I20260811 02:54:38.325712 12843 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDawEwS/build/debug/bin/kudu
/tmp/dist-test-taskDawEwS/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786416707705494-12843-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786416707705494-12843-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786416707705494-12843-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786416707705494-12843-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.138.254:38245
--webserver_interface=127.12.138.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786416707705494-12843-0/krb5kdc/kudu_127.12.138.254.keytab
--principal=kudu/127.12.138.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.138.212:36431
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.12.138.254:38245,127.12.138.253:40553,127.12.138.252:45859
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786416707705494-12843-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786416707705494-12843-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786416707705494-12843-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 02:54:38.442243 28517 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 02:54:38.442559 28517 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 02:54:38.442648 28517 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 02:54:38.446728 28517 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 02:54:38.446815 28517 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 02:54:38.446851 28517 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 02:54:38.446888 28517 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 02:54:38.446929 28517 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 02:54:38.452175 28517 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.138.212:36431
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786416707705494-12843-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786416707705494-12843-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.12.138.254:38245,127.12.138.253:40553,127.12.138.252:45859
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786416707705494-12843-0/krb5kdc/kudu_127.12.138.254.keytab
--principal=kudu/127.12.138.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.138.254:38245
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786416707705494-12843-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.12.138.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28517
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786416707705494-12843-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 02:09:49 UTC on ccycloud.quasar-fsnqqc.root.comops.site
build id 4579570
I20260811 02:54:38.453365 28517 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 02:54:38.454409 28517 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 02:54:38.463577 28523 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:54:38.463553 28522 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:54:38.463608 28527 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:54:38.464002 28517 server_base.cc:1034] running on GCE node
I20260811 02:54:38.464598 28517 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 02:54:38.465793 28517 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 02:54:38.466955 28517 hybrid_clock.cc:648] HybridClock initialized: now 1786416878466937 us; error 38 us; skew 500 ppm
Aug 11 02:54:38 dist-test-slave-9gcw krb5kdc[28504](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416878, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.138.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 02:54:38.469887 28517 init.cc:377] Logged in from keytab as kudu/127.12.138.254@KRBTEST.COM (short username kudu)
I20260811 02:54:38.470818 28517 webserver.cc:460] Webserver started at http://127.12.138.254:44321/ using document root <none> and password file <none>
I20260811 02:54:38.471407 28517 fs_manager.cc:359] Metadata directory not provided
I20260811 02:54:38.471499 28517 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 02:54:38.471737 28517 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 02:54:38.473909 28517 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786416707705494-12843-0/minicluster-data/master-0/data/instance:
uuid: "8a4b4a75a292410bb861f29bb3e7837b"
format_stamp: "Formatted at 2026-08-11 02:54:38 on dist-test-slave-9gcw"
server_key: "faf23e7afe3853a6b441feca766e3ac1"
server_key_iv: "730bd2dd6e3cf0a4e009a7c74d620145"
server_key_version: "encryptionkey@0"
I20260811 02:54:38.474488 28517 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786416707705494-12843-0/minicluster-data/master-0/wal/instance:
uuid: "8a4b4a75a292410bb861f29bb3e7837b"
format_stamp: "Formatted at 2026-08-11 02:54:38 on dist-test-slave-9gcw"
server_key: "faf23e7afe3853a6b441feca766e3ac1"
server_key_iv: "730bd2dd6e3cf0a4e009a7c74d620145"
server_key_version: "encryptionkey@0"
I20260811 02:54:38.478312 28517 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.001s
I20260811 02:54:38.480834 28538 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 02:54:38.482064 28517 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260811 02:54:38.482203 28517 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786416707705494-12843-0/minicluster-data/master-0/data,/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786416707705494-12843-0/minicluster-data/master-0/wal
uuid: "8a4b4a75a292410bb861f29bb3e7837b"
format_stamp: "Formatted at 2026-08-11 02:54:38 on dist-test-slave-9gcw"
server_key: "faf23e7afe3853a6b441feca766e3ac1"
server_key_iv: "730bd2dd6e3cf0a4e009a7c74d620145"
server_key_version: "encryptionkey@0"
I20260811 02:54:38.482336 28517 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786416707705494-12843-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786416707705494-12843-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786416707705494-12843-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 02:54:38.511667 28517 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 02:54:38.514842 28517 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 02:54:38.515058 28517 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 02:54:38.522248 28517 rpc_server.cc:307] RPC server started. Bound to: 127.12.138.254:38245
I20260811 02:54:38.522544 28598 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.138.254:38245 every 8 connection(s)
I20260811 02:54:38.523239 28517 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786416707705494-12843-0/minicluster-data/master-0/data/info.pb
I20260811 02:54:38.523597 12843 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDawEwS/build/debug/bin/kudu as pid 28517
I20260811 02:54:38.523800 12843 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786416707705494-12843-0/minicluster-data/master-0/wal/instance
I20260811 02:54:38.524132 12843 external_mini_cluster.cc:1428] Setting key d0d81450d412798c9e6bd4e05c4410eb
I20260811 02:54:38.526198 28599 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 02:54:38.529959 28599 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.12.138.254" port: 38245 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.12.138.253@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.138.253@KRBTEST.COM" created.
Aug 11 02:54:38 dist-test-slave-9gcw krb5kdc[28504](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416878, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.138.254@KRBTEST.COM for kudu/127.12.138.254@KRBTEST.COM
I20260811 02:54:38.546833 28602 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:54:38.532198 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.138.254:38245 (local address 127.0.0.1:35626)
0811 02:54:38.532781 (+ 583us) negotiation.cc:107] Waiting for socket to connect
0811 02:54:38.532840 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0811 02:54:38.533801 (+ 961us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:54:38.534444 (+ 643us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:54:38.534449 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:54:38.534985 (+ 536us) client_negotiation.cc:189] Negotiated authn=SASL
0811 02:54:38.535253 (+ 268us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:54:38.535266 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:54:38.536625 (+ 1359us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:54:38.536632 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:54:38.537861 (+ 1229us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:54:38.537872 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:54:38.538806 (+ 934us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:54:38.538809 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:54:38.538905 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:54:38.539517 (+ 612us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 02:54:38.539533 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0811 02:54:38.541536 (+ 2003us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 02:54:38.543752 (+ 2216us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:54:38.543763 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:54:38.543776 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:54:38.544105 (+ 329us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:54:38.544444 (+ 339us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:54:38.544448 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:54:38.544450 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:54:38.544619 (+ 169us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:54:38.545357 (+ 738us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 02:54:38.545366 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 02:54:38.545544 (+ 178us) client_negotiation.cc:769] Sending connection context
0811 02:54:38.545725 (+ 181us) client_negotiation.cc:240] Negotiation successful
0811 02:54:38.546006 (+ 281us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":319,"thread_start_us":141,"threads_started":1}
I20260811 02:54:38.546836 28603 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:54:38.532198 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35626 (local address 127.12.138.254:38245)
0811 02:54:38.532729 (+ 531us) server_negotiation.cc:206] Beginning negotiation
0811 02:54:38.532747 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0811 02:54:38.532964 (+ 217us) server_negotiation.cc:407] Connection header received
0811 02:54:38.534076 (+ 1112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:54:38.534080 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:54:38.534252 (+ 172us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:54:38.534406 (+ 154us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:54:38.535487 (+ 1081us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:54:38.536497 (+ 1010us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:54:38.537983 (+ 1486us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:54:38.538625 (+ 642us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:54:38.538805 (+ 180us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:54:38.541686 (+ 2881us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:54:38.541692 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:54:38.541702 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:54:38.541723 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:54:38.543561 (+ 1838us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:54:38.544227 (+ 666us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:54:38.544232 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:54:38.544233 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:54:38.544343 (+ 110us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:54:38.544747 (+ 404us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:54:38.544749 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:54:38.544750 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:54:38.545231 (+ 481us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:54:38.545352 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0811 02:54:38.545831 (+ 479us) server_negotiation.cc:299] Negotiation successful
0811 02:54:38.546080 (+ 249us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":78,"server-negotiator.queue_time_us":410,"spinlock_wait_cycles":27648,"thread_start_us":158,"threads_started":1}
I20260811 02:54:38.549477 28599 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.12.138.253" port: 40553 } has no permanent_uuid. Determining permanent_uuid...
I20260811 02:54:38.550480 28602 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:54:38.550134 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.138.253:40553 (local address 127.0.0.1:56972)
0811 02:54:38.550300 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0811 02:54:38.550393 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.138.253:40553: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":87}
W20260811 02:54:38.550833 28542 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.12.138.253:40553: connect: Connection refused (error 111)
I20260811 02:54:38.551950 28602 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:54:38.551709 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.138.253:40553 (local address 127.0.0.1:56980)
0811 02:54:38.551846 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0811 02:54:38.551896 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.138.253:40553: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":61}
W20260811 02:54:38.552127 28599 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.12.138.253:40553: Network error: Client connection negotiation failed: client connection to 127.12.138.253:40553: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.138.253 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786416707705494-12843-0/krb5kdc/kudu_127.12.138.253.keytab.
Entry for principal kudu/127.12.138.253 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786416707705494-12843-0/krb5kdc/kudu_127.12.138.253.keytab.
I20260811 02:54:38.556049 12843 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDawEwS/build/debug/bin/kudu
/tmp/dist-test-taskDawEwS/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786416707705494-12843-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786416707705494-12843-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786416707705494-12843-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786416707705494-12843-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.138.253:40553
--webserver_interface=127.12.138.253
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786416707705494-12843-0/krb5kdc/kudu_127.12.138.253.keytab
--principal=kudu/127.12.138.253
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.138.212:36431
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.12.138.254:38245,127.12.138.253:40553,127.12.138.252:45859
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786416707705494-12843-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786416707705494-12843-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786416707705494-12843-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 02:54:38.601409 28599 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.12.138.253" port: 40553 } attempt: 1
I20260811 02:54:38.602388 28602 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:54:38.602074 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.138.253:40553 (local address 127.0.0.1:56992)
0811 02:54:38.602229 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0811 02:54:38.602308 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.138.253:40553: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
I20260811 02:54:38.603101 28602 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:54:38.602896 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.138.253:40553 (local address 127.0.0.1:56998)
0811 02:54:38.602999 (+ 103us) negotiation.cc:107] Waiting for socket to connect
0811 02:54:38.603049 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.138.253:40553: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
W20260811 02:54:38.603227 28599 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.12.138.253:40553: Network error: Client connection negotiation failed: client connection to 127.12.138.253:40553: connect: Connection refused (error 111)
I20260811 02:54:38.671402 28599 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.12.138.253" port: 40553 } attempt: 2
I20260811 02:54:38.672432 28602 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:54:38.672075 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.138.253:40553 (local address 127.0.0.1:57008)
0811 02:54:38.672271 (+ 196us) negotiation.cc:107] Waiting for socket to connect
0811 02:54:38.672343 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.138.253:40553: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":87}
I20260811 02:54:38.673328 28602 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:54:38.673057 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.138.253:40553 (local address 127.0.0.1:57016)
0811 02:54:38.673224 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0811 02:54:38.673271 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.138.253:40553: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
W20260811 02:54:38.673462 28599 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.12.138.253:40553: Network error: Client connection negotiation failed: client connection to 127.12.138.253:40553: connect: Connection refused (error 111)
W20260811 02:54:38.685781 28606 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 02:54:38.686107 28606 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 02:54:38.686219 28606 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 02:54:38.691093 28606 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 02:54:38.691187 28606 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 02:54:38.691228 28606 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 02:54:38.691285 28606 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 02:54:38.691329 28606 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 02:54:38.697435 28606 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.138.212:36431
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786416707705494-12843-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786416707705494-12843-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.12.138.254:38245,127.12.138.253:40553,127.12.138.252:45859
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786416707705494-12843-0/krb5kdc/kudu_127.12.138.253.keytab
--principal=kudu/127.12.138.253
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.138.253:40553
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786416707705494-12843-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.12.138.253
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28606
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786416707705494-12843-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 02:09:49 UTC on ccycloud.quasar-fsnqqc.root.comops.site
build id 4579570
I20260811 02:54:38.698662 28606 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 02:54:38.699841 28606 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 02:54:38.709117 28612 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:54:38.709090 28613 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:54:38.709381 28606 server_base.cc:1034] running on GCE node
W20260811 02:54:38.709486 28615 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:54:38.710099 28606 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 02:54:38.711313 28606 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 02:54:38.712527 28606 hybrid_clock.cc:648] HybridClock initialized: now 1786416878712491 us; error 49 us; skew 500 ppm
Aug 11 02:54:38 dist-test-slave-9gcw krb5kdc[28504](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416878, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.138.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 02:54:38.715801 28606 init.cc:377] Logged in from keytab as kudu/127.12.138.253@KRBTEST.COM (short username kudu)
I20260811 02:54:38.716809 28606 webserver.cc:460] Webserver started at http://127.12.138.253:36735/ using document root <none> and password file <none>
I20260811 02:54:38.717521 28606 fs_manager.cc:359] Metadata directory not provided
I20260811 02:54:38.717624 28606 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 02:54:38.717880 28606 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 02:54:38.720152 28606 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786416707705494-12843-0/minicluster-data/master-1/data/instance:
uuid: "e912c1f4612b4e2aab6e32e1f1bf33f0"
format_stamp: "Formatted at 2026-08-11 02:54:38 on dist-test-slave-9gcw"
server_key: "f88621e6f442598a25bd446634ddd6d5"
server_key_iv: "bfe9016c06e9703ad9e7a2340ba20d59"
server_key_version: "encryptionkey@0"
I20260811 02:54:38.720824 28606 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786416707705494-12843-0/minicluster-data/master-1/wal/instance:
uuid: "e912c1f4612b4e2aab6e32e1f1bf33f0"
format_stamp: "Formatted at 2026-08-11 02:54:38 on dist-test-slave-9gcw"
server_key: "f88621e6f442598a25bd446634ddd6d5"
server_key_iv: "bfe9016c06e9703ad9e7a2340ba20d59"
server_key_version: "encryptionkey@0"
I20260811 02:54:38.725205 28606 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.005s
I20260811 02:54:38.728116 28625 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 02:54:38.729605 28606 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 02:54:38.729780 28606 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786416707705494-12843-0/minicluster-data/master-1/data,/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786416707705494-12843-0/minicluster-data/master-1/wal
uuid: "e912c1f4612b4e2aab6e32e1f1bf33f0"
format_stamp: "Formatted at 2026-08-11 02:54:38 on dist-test-slave-9gcw"
server_key: "f88621e6f442598a25bd446634ddd6d5"
server_key_iv: "bfe9016c06e9703ad9e7a2340ba20d59"
server_key_version: "encryptionkey@0"
I20260811 02:54:38.729915 28606 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786416707705494-12843-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786416707705494-12843-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786416707705494-12843-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 02:54:38.747635 28606 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 02:54:38.751394 28606 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 02:54:38.751633 28606 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 02:54:38.760041 28606 rpc_server.cc:307] RPC server started. Bound to: 127.12.138.253:40553
I20260811 02:54:38.760344 28687 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.138.253:40553 every 8 connection(s)
I20260811 02:54:38.761197 28606 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786416707705494-12843-0/minicluster-data/master-1/data/info.pb
I20260811 02:54:38.764047 28688 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 02:54:38.764521 12843 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDawEwS/build/debug/bin/kudu as pid 28606
I20260811 02:54:38.764635 28599 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.12.138.253" port: 40553 } attempt: 3
I20260811 02:54:38.764786 12843 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786416707705494-12843-0/minicluster-data/master-1/wal/instance
I20260811 02:54:38.765269 12843 external_mini_cluster.cc:1428] Setting key d2ac0bccde6873a00f976e4c1ef7fcff
I20260811 02:54:38.770105 28688 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.12.138.254" port: 38245 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 02:54:38 dist-test-slave-9gcw krb5kdc[28504](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416878, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.138.254@KRBTEST.COM for kudu/127.12.138.253@KRBTEST.COM
WARNING: no policy specified for kudu/127.12.138.252@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.138.252@KRBTEST.COM" created.
I20260811 02:54:38.779978 28602 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:54:38.765276 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.138.253:40553 (local address 127.0.0.1:57024)
0811 02:54:38.765504 (+ 228us) negotiation.cc:107] Waiting for socket to connect
0811 02:54:38.765524 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0811 02:54:38.765631 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:54:38.768519 (+ 2888us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:54:38.768523 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:54:38.768778 (+ 255us) client_negotiation.cc:189] Negotiated authn=SASL
0811 02:54:38.768942 (+ 164us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:54:38.768951 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:54:38.770447 (+ 1496us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:54:38.770454 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:54:38.771150 (+ 696us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:54:38.771163 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:54:38.772142 (+ 979us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:54:38.772147 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:54:38.772299 (+ 152us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:54:38.772872 (+ 573us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 02:54:38.772893 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0811 02:54:38.774667 (+ 1774us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 02:54:38.777466 (+ 2799us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:54:38.777469 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:54:38.777471 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:54:38.777739 (+ 268us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:54:38.778119 (+ 380us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:54:38.778121 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:54:38.778123 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:54:38.778238 (+ 115us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:54:38.779227 (+ 989us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 02:54:38.779230 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 02:54:38.779350 (+ 120us) client_negotiation.cc:769] Sending connection context
0811 02:54:38.779479 (+ 129us) client_negotiation.cc:240] Negotiation successful
0811 02:54:38.779714 (+ 235us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":136}
Aug 11 02:54:38 dist-test-slave-9gcw krb5kdc[28504](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416878, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.138.253@KRBTEST.COM for kudu/127.12.138.254@KRBTEST.COM
I20260811 02:54:38.781139 28690 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:54:38.765944 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57024 (local address 127.12.138.253:40553)
0811 02:54:38.766463 (+ 519us) server_negotiation.cc:206] Beginning negotiation
0811 02:54:38.766488 (+ 25us) server_negotiation.cc:399] Waiting for connection header
0811 02:54:38.766538 (+ 50us) server_negotiation.cc:407] Connection header received
0811 02:54:38.767820 (+ 1282us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:54:38.767849 (+ 29us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:54:38.768220 (+ 371us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:54:38.768593 (+ 373us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:54:38.769084 (+ 491us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:54:38.770278 (+ 1194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:54:38.771355 (+ 1077us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:54:38.771892 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:54:38.772162 (+ 270us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:54:38.774877 (+ 2715us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:54:38.774885 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:54:38.774899 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:54:38.774934 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:54:38.777316 (+ 2382us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:54:38.777843 (+ 527us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:54:38.777849 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:54:38.777856 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:54:38.778016 (+ 160us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:54:38.778346 (+ 330us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:54:38.778349 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:54:38.778350 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:54:38.779061 (+ 711us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:54:38.779315 (+ 254us) server_negotiation.cc:1035] Waiting for connection context
0811 02:54:38.779670 (+ 355us) server_negotiation.cc:299] Negotiation successful
0811 02:54:38.780018 (+ 348us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":325,"thread_start_us":143,"threads_started":1}
I20260811 02:54:38.783722 28599 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.12.138.252" port: 45859 } has no permanent_uuid. Determining permanent_uuid...
I20260811 02:54:38.784485 28602 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:54:38.784219 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.138.252:45859 (local address 127.0.0.1:37070)
0811 02:54:38.784338 (+ 119us) negotiation.cc:107] Waiting for socket to connect
0811 02:54:38.784392 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.138.252:45859: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
I20260811 02:54:38.785419 28602 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:54:38.785042 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.138.252:45859 (local address 127.0.0.1:37086)
0811 02:54:38.785270 (+ 228us) negotiation.cc:107] Waiting for socket to connect
0811 02:54:38.785339 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.138.252:45859: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":142}
W20260811 02:54:38.785645 28599 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.12.138.252:45859: Network error: Client connection negotiation failed: client connection to 127.12.138.252:45859: connect: Connection refused (error 111)
I20260811 02:54:38.785677 28603 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:54:38.772530 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35642 (local address 127.12.138.254:38245)
0811 02:54:38.772678 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0811 02:54:38.772683 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 02:54:38.773397 (+ 714us) server_negotiation.cc:407] Connection header received
0811 02:54:38.773619 (+ 222us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:54:38.773622 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:54:38.773682 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:54:38.773730 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:54:38.774598 (+ 868us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:54:38.775468 (+ 870us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:54:38.777274 (+ 1806us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:54:38.777667 (+ 393us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:54:38.777796 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:54:38.780651 (+ 2855us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:54:38.780658 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:54:38.780664 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:54:38.780696 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:54:38.783283 (+ 2587us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:54:38.783874 (+ 591us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:54:38.783879 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:54:38.783883 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:54:38.784000 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:54:38.784446 (+ 446us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:54:38.784450 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:54:38.784452 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:54:38.784772 (+ 320us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:54:38.784884 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0811 02:54:38.785359 (+ 475us) server_negotiation.cc:299] Negotiation successful
0811 02:54:38.785507 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260811 02:54:38.785754 28693 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:54:38.772484 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.138.254:38245 (local address 127.0.0.1:35642)
0811 02:54:38.773264 (+ 780us) negotiation.cc:107] Waiting for socket to connect
0811 02:54:38.773320 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0811 02:54:38.773488 (+ 168us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:54:38.773893 (+ 405us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:54:38.773897 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:54:38.774254 (+ 357us) client_negotiation.cc:189] Negotiated authn=SASL
0811 02:54:38.774469 (+ 215us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:54:38.774477 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:54:38.775923 (+ 1446us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:54:38.775926 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:54:38.777068 (+ 1142us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:54:38.777083 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:54:38.777835 (+ 752us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:54:38.777839 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:54:38.777976 (+ 137us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:54:38.778519 (+ 543us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 02:54:38.778535 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0811 02:54:38.780504 (+ 1969us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 02:54:38.783399 (+ 2895us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:54:38.783410 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:54:38.783416 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:54:38.783746 (+ 330us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:54:38.784164 (+ 418us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:54:38.784167 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:54:38.784169 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:54:38.784286 (+ 117us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:54:38.784908 (+ 622us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 02:54:38.784918 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 02:54:38.785074 (+ 156us) client_negotiation.cc:769] Sending connection context
0811 02:54:38.785284 (+ 210us) client_negotiation.cc:240] Negotiation successful
0811 02:54:38.785507 (+ 223us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":561,"thread_start_us":467,"threads_started":1}
I20260811 02:54:38.787094 28688 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.12.138.253" port: 40553 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 02:54:38 dist-test-slave-9gcw krb5kdc[28504](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416878, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.138.253@KRBTEST.COM for kudu/127.12.138.253@KRBTEST.COM
I20260811 02:54:38.798242 28693 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:54:38.787583 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.138.253:40553 (local address 127.0.0.1:57038)
0811 02:54:38.787771 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0811 02:54:38.787785 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 02:54:38.787863 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:54:38.788078 (+ 215us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:54:38.788081 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:54:38.788293 (+ 212us) client_negotiation.cc:189] Negotiated authn=SASL
0811 02:54:38.788380 (+ 87us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:54:38.788386 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:54:38.789098 (+ 712us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:54:38.789104 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:54:38.789676 (+ 572us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:54:38.789685 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:54:38.790138 (+ 453us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:54:38.790141 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:54:38.790241 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:54:38.790772 (+ 531us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 02:54:38.790783 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0811 02:54:38.794500 (+ 3717us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 02:54:38.796504 (+ 2004us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:54:38.796511 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:54:38.796514 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:54:38.796732 (+ 218us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:54:38.797053 (+ 321us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:54:38.797056 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:54:38.797058 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:54:38.797213 (+ 155us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:54:38.797675 (+ 462us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 02:54:38.797679 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 02:54:38.797778 (+ 99us) client_negotiation.cc:769] Sending connection context
0811 02:54:38.797891 (+ 113us) client_negotiation.cc:240] Negotiation successful
0811 02:54:38.798046 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":130}
I20260811 02:54:38.798259 28690 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:54:38.787672 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57038 (local address 127.12.138.253:40553)
0811 02:54:38.787864 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0811 02:54:38.787868 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 02:54:38.787880 (+ 12us) server_negotiation.cc:407] Connection header received
0811 02:54:38.787943 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:54:38.787947 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:54:38.787982 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:54:38.788067 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:54:38.788487 (+ 420us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:54:38.788998 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:54:38.789788 (+ 790us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:54:38.790034 (+ 246us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:54:38.790143 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:54:38.794745 (+ 4602us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:54:38.794748 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:54:38.794750 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:54:38.794773 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:54:38.796383 (+ 1610us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:54:38.796863 (+ 480us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:54:38.796866 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:54:38.796869 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:54:38.796948 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:54:38.797346 (+ 398us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:54:38.797349 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:54:38.797350 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:54:38.797564 (+ 214us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:54:38.797677 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0811 02:54:38.798018 (+ 341us) server_negotiation.cc:299] Negotiation successful
0811 02:54:38.798127 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":137}
I20260811 02:54:38.799141 28688 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.12.138.252" port: 45859 } has no permanent_uuid. Determining permanent_uuid...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.138.252 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786416707705494-12843-0/krb5kdc/kudu_127.12.138.252.keytab.
Entry for principal kudu/127.12.138.252 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786416707705494-12843-0/krb5kdc/kudu_127.12.138.252.keytab.
I20260811 02:54:38.800045 28693 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:54:38.799764 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.138.252:45859 (local address 127.0.0.1:37092)
0811 02:54:38.799893 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0811 02:54:38.799990 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.138.252:45859: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
W20260811 02:54:38.800681 28628 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.12.138.252:45859: connect: Connection refused (error 111)
I20260811 02:54:38.800843 12843 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDawEwS/build/debug/bin/kudu
/tmp/dist-test-taskDawEwS/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786416707705494-12843-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786416707705494-12843-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786416707705494-12843-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786416707705494-12843-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.138.252:45859
--webserver_interface=127.12.138.252
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786416707705494-12843-0/krb5kdc/kudu_127.12.138.252.keytab
--principal=kudu/127.12.138.252
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.138.212:36431
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.12.138.254:38245,127.12.138.253:40553,127.12.138.252:45859
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786416707705494-12843-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786416707705494-12843-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786416707705494-12843-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 02:54:38.801952 28693 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:54:38.801600 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.138.252:45859 (local address 127.0.0.1:37104)
0811 02:54:38.801823 (+ 223us) negotiation.cc:107] Waiting for socket to connect
0811 02:54:38.801878 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.138.252:45859: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":68}
W20260811 02:54:38.802134 28688 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.12.138.252:45859: Network error: Client connection negotiation failed: client connection to 127.12.138.252:45859: connect: Connection refused (error 111)
I20260811 02:54:38.816803 28599 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.12.138.252" port: 45859 } attempt: 1
I20260811 02:54:38.817763 28602 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:54:38.817417 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.138.252:45859 (local address 127.0.0.1:37110)
0811 02:54:38.817585 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0811 02:54:38.817662 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.138.252:45859: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
I20260811 02:54:38.818573 28602 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:54:38.818337 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.138.252:45859 (local address 127.0.0.1:37120)
0811 02:54:38.818454 (+ 117us) negotiation.cc:107] Waiting for socket to connect
0811 02:54:38.818510 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.138.252:45859: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
W20260811 02:54:38.818764 28599 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.12.138.252:45859: Network error: Client connection negotiation failed: client connection to 127.12.138.252:45859: connect: Connection refused (error 111)
I20260811 02:54:38.851444 28688 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.12.138.252" port: 45859 } attempt: 1
I20260811 02:54:38.852348 28693 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:54:38.852045 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.138.252:45859 (local address 127.0.0.1:37128)
0811 02:54:38.852198 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0811 02:54:38.852270 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.138.252:45859: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
I20260811 02:54:38.853049 28693 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:54:38.852837 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.138.252:45859 (local address 127.0.0.1:37140)
0811 02:54:38.852942 (+ 105us) negotiation.cc:107] Waiting for socket to connect
0811 02:54:38.852993 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.138.252:45859: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
W20260811 02:54:38.853222 28688 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.12.138.252:45859: Network error: Client connection negotiation failed: client connection to 127.12.138.252:45859: connect: Connection refused (error 111)
I20260811 02:54:38.893957 28599 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.12.138.252" port: 45859 } attempt: 2
I20260811 02:54:38.895183 28602 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:54:38.894716 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.138.252:45859 (local address 127.0.0.1:37152)
0811 02:54:38.894979 (+ 263us) negotiation.cc:107] Waiting for socket to connect
0811 02:54:38.895062 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.138.252:45859: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":120}
I20260811 02:54:38.896055 28602 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:54:38.895795 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.138.252:45859 (local address 127.0.0.1:37166)
0811 02:54:38.895920 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0811 02:54:38.895983 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.138.252:45859: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
W20260811 02:54:38.896207 28599 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.12.138.252:45859: Network error: Client connection negotiation failed: client connection to 127.12.138.252:45859: connect: Connection refused (error 111)
I20260811 02:54:38.921432 28688 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.12.138.252" port: 45859 } attempt: 2
I20260811 02:54:38.922413 28693 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:54:38.922060 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.138.252:45859 (local address 127.0.0.1:37176)
0811 02:54:38.922250 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0811 02:54:38.922327 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.138.252:45859: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":85}
I20260811 02:54:38.924212 28693 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:54:38.923859 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.138.252:45859 (local address 127.0.0.1:37190)
0811 02:54:38.924005 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0811 02:54:38.924095 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.138.252:45859: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
W20260811 02:54:38.924479 28688 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.12.138.252:45859: Network error: Client connection negotiation failed: client connection to 127.12.138.252:45859: connect: Connection refused (error 111)
W20260811 02:54:38.931602 28696 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 02:54:38.932044 28696 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 02:54:38.932132 28696 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 02:54:38.936969 28696 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 02:54:38.937083 28696 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 02:54:38.937112 28696 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 02:54:38.937139 28696 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 02:54:38.937192 28696 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 02:54:38.943159 28696 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.138.212:36431
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786416707705494-12843-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786416707705494-12843-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.12.138.254:38245,127.12.138.253:40553,127.12.138.252:45859
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786416707705494-12843-0/krb5kdc/kudu_127.12.138.252.keytab
--principal=kudu/127.12.138.252
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.138.252:45859
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786416707705494-12843-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.12.138.252
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28696
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786416707705494-12843-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 02:09:49 UTC on ccycloud.quasar-fsnqqc.root.comops.site
build id 4579570
I20260811 02:54:38.944357 28696 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 02:54:38.945547 28696 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260811 02:54:38.954710 28696 server_base.cc:1034] running on GCE node
W20260811 02:54:38.954671 28704 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:54:38.954672 28707 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:54:38.954671 28705 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:54:38.955591 28696 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 02:54:38.956763 28696 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 02:54:38.957957 28696 hybrid_clock.cc:648] HybridClock initialized: now 1786416878957935 us; error 40 us; skew 500 ppm
Aug 11 02:54:38 dist-test-slave-9gcw krb5kdc[28504](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416878, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.138.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 02:54:38.961004 28696 init.cc:377] Logged in from keytab as kudu/127.12.138.252@KRBTEST.COM (short username kudu)
I20260811 02:54:38.961984 28696 webserver.cc:460] Webserver started at http://127.12.138.252:37133/ using document root <none> and password file <none>
I20260811 02:54:38.962563 28696 fs_manager.cc:359] Metadata directory not provided
I20260811 02:54:38.962630 28696 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 02:54:38.962862 28696 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 02:54:38.965135 28696 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786416707705494-12843-0/minicluster-data/master-2/data/instance:
uuid: "3fbdf7973fac4c11895279823cfcd3cf"
format_stamp: "Formatted at 2026-08-11 02:54:38 on dist-test-slave-9gcw"
server_key: "969a52baf53effb7c853e28d0392ec14"
server_key_iv: "3c48ec3e76bb5889ea4610c66a6bf581"
server_key_version: "encryptionkey@0"
I20260811 02:54:38.965814 28696 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786416707705494-12843-0/minicluster-data/master-2/wal/instance:
uuid: "3fbdf7973fac4c11895279823cfcd3cf"
format_stamp: "Formatted at 2026-08-11 02:54:38 on dist-test-slave-9gcw"
server_key: "969a52baf53effb7c853e28d0392ec14"
server_key_iv: "3c48ec3e76bb5889ea4610c66a6bf581"
server_key_version: "encryptionkey@0"
I20260811 02:54:38.969897 28696 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260811 02:54:38.972596 28716 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 02:54:38.974037 28696 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260811 02:54:38.974155 28696 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786416707705494-12843-0/minicluster-data/master-2/data,/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786416707705494-12843-0/minicluster-data/master-2/wal
uuid: "3fbdf7973fac4c11895279823cfcd3cf"
format_stamp: "Formatted at 2026-08-11 02:54:38 on dist-test-slave-9gcw"
server_key: "969a52baf53effb7c853e28d0392ec14"
server_key_iv: "3c48ec3e76bb5889ea4610c66a6bf581"
server_key_version: "encryptionkey@0"
I20260811 02:54:38.974310 28696 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786416707705494-12843-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786416707705494-12843-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786416707705494-12843-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 02:54:38.988560 28696 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 02:54:38.992069 28696 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 02:54:38.992282 28696 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 02:54:38.995383 28599 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.12.138.252" port: 45859 } attempt: 3
I20260811 02:54:38.996343 28602 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:54:38.995901 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.138.252:45859 (local address 127.0.0.1:37194)
0811 02:54:38.996149 (+ 248us) negotiation.cc:107] Waiting for socket to connect
0811 02:54:38.996221 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.138.252:45859: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":148}
I20260811 02:54:38.997334 28602 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:54:38.997003 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.138.252:45859 (local address 127.0.0.1:37204)
0811 02:54:38.997169 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0811 02:54:38.997240 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.138.252:45859: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
W20260811 02:54:38.997485 28599 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.12.138.252:45859: Network error: Client connection negotiation failed: client connection to 127.12.138.252:45859: connect: Connection refused (error 111)
I20260811 02:54:39.000265 28696 rpc_server.cc:307] RPC server started. Bound to: 127.12.138.252:45859
I20260811 02:54:39.000582 28774 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.138.252:45859 every 8 connection(s)
I20260811 02:54:39.001400 28696 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786416707705494-12843-0/minicluster-data/master-2/data/info.pb
I20260811 02:54:39.004472 28775 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 02:54:39.008587 28775 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.12.138.254" port: 38245 } has no permanent_uuid. Determining permanent_uuid...
I20260811 02:54:39.010366 12843 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDawEwS/build/debug/bin/kudu as pid 28696
I20260811 02:54:39.010504 12843 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786416707705494-12843-0/minicluster-data/master-2/wal/instance
I20260811 02:54:39.010857 12843 external_mini_cluster.cc:1428] Setting key bcb07890df14d59de279c8a729b8c63e
I20260811 02:54:39.015671 28688 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.12.138.252" port: 45859 } attempt: 3
Aug 11 02:54:39 dist-test-slave-9gcw krb5kdc[28504](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416878, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.138.253@KRBTEST.COM for kudu/127.12.138.252@KRBTEST.COM
Aug 11 02:54:39 dist-test-slave-9gcw krb5kdc[28504](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416878, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.138.252@KRBTEST.COM for kudu/127.12.138.254@KRBTEST.COM
Aug 11 02:54:39 dist-test-slave-9gcw krb5kdc[28504](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416878, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.12.138.252@KRBTEST.COM
I20260811 02:54:39.027972 28693 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:54:39.016158 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.138.252:45859 (local address 127.0.0.1:37218)
0811 02:54:39.016360 (+ 202us) negotiation.cc:107] Waiting for socket to connect
0811 02:54:39.016375 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 02:54:39.016448 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:54:39.016972 (+ 524us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:54:39.016975 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:54:39.017232 (+ 257us) client_negotiation.cc:189] Negotiated authn=SASL
0811 02:54:39.017340 (+ 108us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:54:39.017347 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:54:39.018189 (+ 842us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:54:39.018192 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:54:39.018715 (+ 523us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:54:39.018726 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:54:39.019156 (+ 430us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:54:39.019159 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:54:39.019233 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:54:39.021297 (+ 2064us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 02:54:39.021313 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0811 02:54:39.022954 (+ 1641us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 02:54:39.025691 (+ 2737us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:54:39.025696 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:54:39.025700 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:54:39.026022 (+ 322us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:54:39.026440 (+ 418us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:54:39.026443 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:54:39.026445 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:54:39.026557 (+ 112us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:54:39.027385 (+ 828us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 02:54:39.027388 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 02:54:39.027499 (+ 111us) client_negotiation.cc:769] Sending connection context
0811 02:54:39.027590 (+ 91us) client_negotiation.cc:240] Negotiation successful
0811 02:54:39.027779 (+ 189us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":120}
I20260811 02:54:39.028045 28603 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:54:39.010730 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35654 (local address 127.12.138.254:38245)
0811 02:54:39.010905 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0811 02:54:39.010911 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 02:54:39.011868 (+ 957us) server_negotiation.cc:407] Connection header received
0811 02:54:39.013067 (+ 1199us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:54:39.013070 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:54:39.013120 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:54:39.013267 (+ 147us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:54:39.014245 (+ 978us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:54:39.014765 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:54:39.015952 (+ 1187us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:54:39.016272 (+ 320us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:54:39.016365 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:54:39.023740 (+ 7375us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:54:39.023745 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:54:39.023747 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:54:39.023782 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:54:39.025760 (+ 1978us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:54:39.026364 (+ 604us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:54:39.026368 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:54:39.026369 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:54:39.026468 (+ 99us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:54:39.026917 (+ 449us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:54:39.026920 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:54:39.026922 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:54:39.027148 (+ 226us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:54:39.027267 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0811 02:54:39.027775 (+ 508us) server_negotiation.cc:299] Negotiation successful
0811 02:54:39.027902 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260811 02:54:39.029634 28780 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:54:39.011592 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37210 (local address 127.12.138.252:45859)
0811 02:54:39.012266 (+ 674us) server_negotiation.cc:206] Beginning negotiation
0811 02:54:39.012277 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 02:54:39.012318 (+ 41us) server_negotiation.cc:407] Connection header received
0811 02:54:39.012836 (+ 518us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:54:39.012860 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:54:39.013113 (+ 253us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:54:39.013370 (+ 257us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:54:39.014106 (+ 736us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:54:39.015113 (+ 1007us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:54:39.016097 (+ 984us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:54:39.016693 (+ 596us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:54:39.016796 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:54:39.024336 (+ 7540us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:54:39.024341 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:54:39.024344 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:54:39.024373 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:54:39.026710 (+ 2337us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:54:39.027175 (+ 465us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:54:39.027180 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:54:39.027182 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:54:39.027270 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:54:39.027540 (+ 270us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:54:39.027545 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:54:39.027547 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:54:39.027719 (+ 172us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:54:39.027959 (+ 240us) server_negotiation.cc:1035] Waiting for connection context
0811 02:54:39.028642 (+ 683us) server_negotiation.cc:299] Negotiation successful
0811 02:54:39.028742 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":26,"server-negotiator.queue_time_us":587,"spinlock_wait_cycles":4480,"thread_start_us":157,"threads_started":1}
I20260811 02:54:39.029641 28778 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:54:39.011149 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.138.254:38245 (local address 127.0.0.1:35654)
0811 02:54:39.011704 (+ 555us) negotiation.cc:107] Waiting for socket to connect
0811 02:54:39.011758 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0811 02:54:39.012834 (+ 1076us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:54:39.013343 (+ 509us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:54:39.013349 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:54:39.013798 (+ 449us) client_negotiation.cc:189] Negotiated authn=SASL
0811 02:54:39.014084 (+ 286us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:54:39.014104 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:54:39.014877 (+ 773us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:54:39.014880 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:54:39.015777 (+ 897us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:54:39.015787 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:54:39.016548 (+ 761us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:54:39.016551 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:54:39.016756 (+ 205us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:54:39.021378 (+ 4622us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 02:54:39.021402 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0811 02:54:39.023552 (+ 2150us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 02:54:39.025875 (+ 2323us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:54:39.025887 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:54:39.025898 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:54:39.026239 (+ 341us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:54:39.026584 (+ 345us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:54:39.026590 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:54:39.026595 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:54:39.026742 (+ 147us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:54:39.027325 (+ 583us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 02:54:39.027334 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 02:54:39.027495 (+ 161us) client_negotiation.cc:769] Sending connection context
0811 02:54:39.028262 (+ 767us) client_negotiation.cc:240] Negotiation successful
0811 02:54:39.028663 (+ 401us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":343,"mutex_wait_us":119,"spinlock_wait_cycles":1408,"thread_start_us":182,"threads_started":1}
I20260811 02:54:39.029641 28781 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:54:39.016249 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37218 (local address 127.12.138.252:45859)
0811 02:54:39.016699 (+ 450us) server_negotiation.cc:206] Beginning negotiation
0811 02:54:39.016709 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 02:54:39.016725 (+ 16us) server_negotiation.cc:407] Connection header received
0811 02:54:39.016771 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:54:39.016773 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:54:39.016819 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:54:39.017436 (+ 617us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:54:39.017507 (+ 71us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:54:39.018053 (+ 546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:54:39.018835 (+ 782us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:54:39.019056 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:54:39.019134 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:54:39.023126 (+ 3992us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:54:39.023135 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:54:39.023155 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:54:39.023190 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:54:39.025553 (+ 2363us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:54:39.026149 (+ 596us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:54:39.026157 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:54:39.026160 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:54:39.026308 (+ 148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:54:39.026648 (+ 340us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:54:39.026652 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:54:39.026654 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:54:39.027258 (+ 604us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:54:39.028197 (+ 939us) server_negotiation.cc:1035] Waiting for connection context
0811 02:54:39.028313 (+ 116us) server_negotiation.cc:299] Negotiation successful
0811 02:54:39.028607 (+ 294us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":377,"thread_start_us":107,"threads_started":1}
I20260811 02:54:39.031075 28775 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.12.138.253" port: 40553 } has no permanent_uuid. Determining permanent_uuid...
I20260811 02:54:39.034766 28688 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e912c1f4612b4e2aab6e32e1f1bf33f0: Bootstrap starting.
Aug 11 02:54:39 dist-test-slave-9gcw krb5kdc[28504](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416878, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.138.252@KRBTEST.COM for kudu/127.12.138.253@KRBTEST.COM
I20260811 02:54:39.037771 28688 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e912c1f4612b4e2aab6e32e1f1bf33f0: Neither blocks nor log segments found. Creating new log.
I20260811 02:54:39.038839 28688 log.cc:824] T 00000000000000000000000000000000 P e912c1f4612b4e2aab6e32e1f1bf33f0: Log is configured to *not* fsync() on all Append() calls
Aug 11 02:54:39 dist-test-slave-9gcw krb5kdc[28504](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416878, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.12.138.254@KRBTEST.COM
I20260811 02:54:39.041535 28688 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e912c1f4612b4e2aab6e32e1f1bf33f0: No bootstrap required, opened a new log
I20260811 02:54:39.042910 28690 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:54:39.031824 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57042 (local address 127.12.138.253:40553)
0811 02:54:39.031943 (+ 119us) server_negotiation.cc:206] Beginning negotiation
0811 02:54:39.031950 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 02:54:39.031999 (+ 49us) server_negotiation.cc:407] Connection header received
0811 02:54:39.032247 (+ 248us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:54:39.032250 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:54:39.032309 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:54:39.032397 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:54:39.032856 (+ 459us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:54:39.033636 (+ 780us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:54:39.034773 (+ 1137us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:54:39.035094 (+ 321us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:54:39.035219 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:54:39.038079 (+ 2860us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:54:39.038084 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:54:39.038087 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:54:39.038120 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:54:39.040008 (+ 1888us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:54:39.041502 (+ 1494us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:54:39.041507 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:54:39.041509 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:54:39.041613 (+ 104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:54:39.041962 (+ 349us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:54:39.041966 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:54:39.041967 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:54:39.042210 (+ 243us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:54:39.042321 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0811 02:54:39.042623 (+ 302us) server_negotiation.cc:299] Negotiation successful
0811 02:54:39.042754 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260811 02:54:39.042955 28778 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:54:39.031771 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.138.253:40553 (local address 127.0.0.1:57042)
0811 02:54:39.031938 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0811 02:54:39.031953 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 02:54:39.032051 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:54:39.032412 (+ 361us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:54:39.032415 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:54:39.032624 (+ 209us) client_negotiation.cc:189] Negotiated authn=SASL
0811 02:54:39.032731 (+ 107us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:54:39.032738 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:54:39.033757 (+ 1019us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:54:39.033762 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:54:39.034612 (+ 850us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:54:39.034622 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:54:39.035231 (+ 609us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:54:39.035235 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:54:39.035348 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:54:39.035913 (+ 565us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 02:54:39.035928 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0811 02:54:39.037900 (+ 1972us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 02:54:39.040421 (+ 2521us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:54:39.040424 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:54:39.040427 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:54:39.041372 (+ 945us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:54:39.041764 (+ 392us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:54:39.041766 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:54:39.041768 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:54:39.041861 (+ 93us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:54:39.042309 (+ 448us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 02:54:39.042312 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 02:54:39.042416 (+ 104us) client_negotiation.cc:769] Sending connection context
0811 02:54:39.042564 (+ 148us) client_negotiation.cc:240] Negotiation successful
0811 02:54:39.042746 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":82}
I20260811 02:54:39.043751 28603 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:54:39.033016 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35670 (local address 127.12.138.254:38245)
0811 02:54:39.033282 (+ 266us) server_negotiation.cc:206] Beginning negotiation
0811 02:54:39.033286 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 02:54:39.033298 (+ 12us) server_negotiation.cc:407] Connection header received
0811 02:54:39.033340 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:54:39.033342 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:54:39.033399 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:54:39.033477 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:54:39.034288 (+ 811us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:54:39.035047 (+ 759us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:54:39.036165 (+ 1118us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:54:39.036571 (+ 406us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:54:39.036645 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:54:39.039576 (+ 2931us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:54:39.039580 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:54:39.039583 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:54:39.039616 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:54:39.041451 (+ 1835us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:54:39.042078 (+ 627us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:54:39.042082 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:54:39.042084 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:54:39.042178 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:54:39.042502 (+ 324us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:54:39.042505 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:54:39.042522 (+ 17us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:54:39.042755 (+ 233us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:54:39.043027 (+ 272us) server_negotiation.cc:1035] Waiting for connection context
0811 02:54:39.043427 (+ 400us) server_negotiation.cc:299] Negotiation successful
0811 02:54:39.043562 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":180}
I20260811 02:54:39.044021 28775 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.12.138.252" port: 45859 } has no permanent_uuid. Determining permanent_uuid...
I20260811 02:54:39.047101 28688 raft_consensus.cc:348] T 00000000000000000000000000000000 P e912c1f4612b4e2aab6e32e1f1bf33f0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a4b4a75a292410bb861f29bb3e7837b" member_type: VOTER last_known_addr { host: "127.12.138.254" port: 38245 } } peers { permanent_uuid: "e912c1f4612b4e2aab6e32e1f1bf33f0" member_type: VOTER last_known_addr { host: "127.12.138.253" port: 40553 } } peers { permanent_uuid: "3fbdf7973fac4c11895279823cfcd3cf" member_type: VOTER last_known_addr { host: "127.12.138.252" port: 45859 } }
I20260811 02:54:39.047345 28688 raft_consensus.cc:374] T 00000000000000000000000000000000 P e912c1f4612b4e2aab6e32e1f1bf33f0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 02:54:39.047405 28688 raft_consensus.cc:729] T 00000000000000000000000000000000 P e912c1f4612b4e2aab6e32e1f1bf33f0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e912c1f4612b4e2aab6e32e1f1bf33f0, State: Initialized, Role: FOLLOWER
I20260811 02:54:39.047966 28688 consensus_queue.cc:260] T 00000000000000000000000000000000 P e912c1f4612b4e2aab6e32e1f1bf33f0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a4b4a75a292410bb861f29bb3e7837b" member_type: VOTER last_known_addr { host: "127.12.138.254" port: 38245 } } peers { permanent_uuid: "e912c1f4612b4e2aab6e32e1f1bf33f0" member_type: VOTER last_known_addr { host: "127.12.138.253" port: 40553 } } peers { permanent_uuid: "3fbdf7973fac4c11895279823cfcd3cf" member_type: VOTER last_known_addr { host: "127.12.138.252" port: 45859 } }
I20260811 02:54:39.048902 28783 sys_catalog.cc:455] T 00000000000000000000000000000000 P e912c1f4612b4e2aab6e32e1f1bf33f0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a4b4a75a292410bb861f29bb3e7837b" member_type: VOTER last_known_addr { host: "127.12.138.254" port: 38245 } } peers { permanent_uuid: "e912c1f4612b4e2aab6e32e1f1bf33f0" member_type: VOTER last_known_addr { host: "127.12.138.253" port: 40553 } } peers { permanent_uuid: "3fbdf7973fac4c11895279823cfcd3cf" member_type: VOTER last_known_addr { host: "127.12.138.252" port: 45859 } } }
I20260811 02:54:39.049062 28783 sys_catalog.cc:458] T 00000000000000000000000000000000 P e912c1f4612b4e2aab6e32e1f1bf33f0 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 02:54:39.049543 28688 sys_catalog.cc:564] T 00000000000000000000000000000000 P e912c1f4612b4e2aab6e32e1f1bf33f0 [sys.catalog]: configured and running, proceeding with master startup.
Aug 11 02:54:39 dist-test-slave-9gcw krb5kdc[28504](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416878, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.138.252@KRBTEST.COM for kudu/127.12.138.252@KRBTEST.COM
Aug 11 02:54:39 dist-test-slave-9gcw krb5kdc[28504](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416878, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.12.138.253@KRBTEST.COM
I20260811 02:54:39.054638 28778 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:54:39.044606 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.138.252:45859 (local address 127.0.0.1:37232)
0811 02:54:39.044702 (+ 96us) negotiation.cc:107] Waiting for socket to connect
0811 02:54:39.044718 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 02:54:39.044792 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:54:39.045359 (+ 567us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:54:39.045362 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:54:39.045539 (+ 177us) client_negotiation.cc:189] Negotiated authn=SASL
0811 02:54:39.045646 (+ 107us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:54:39.045653 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:54:39.046671 (+ 1018us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:54:39.046674 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:54:39.047399 (+ 725us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:54:39.047411 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:54:39.047962 (+ 551us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:54:39.047966 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:54:39.048051 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:54:39.048725 (+ 674us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 02:54:39.048740 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0811 02:54:39.050873 (+ 2133us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 02:54:39.052899 (+ 2026us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:54:39.052902 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:54:39.052904 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:54:39.053193 (+ 289us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:54:39.053544 (+ 351us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:54:39.053548 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:54:39.053549 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:54:39.053641 (+ 92us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:54:39.054125 (+ 484us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 02:54:39.054128 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 02:54:39.054214 (+ 86us) client_negotiation.cc:769] Sending connection context
0811 02:54:39.054315 (+ 101us) client_negotiation.cc:240] Negotiation successful
0811 02:54:39.054463 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":28}
I20260811 02:54:39.055112 28781 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:54:39.044664 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37232 (local address 127.12.138.252:45859)
0811 02:54:39.045093 (+ 429us) server_negotiation.cc:206] Beginning negotiation
0811 02:54:39.045097 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 02:54:39.045108 (+ 11us) server_negotiation.cc:407] Connection header received
0811 02:54:39.045186 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:54:39.045194 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:54:39.045241 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:54:39.045327 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:54:39.045768 (+ 441us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:54:39.046555 (+ 787us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:54:39.047530 (+ 975us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:54:39.047838 (+ 308us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:54:39.047938 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:54:39.051045 (+ 3107us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:54:39.051053 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:54:39.051059 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:54:39.051091 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:54:39.052756 (+ 1665us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:54:39.053300 (+ 544us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:54:39.053303 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:54:39.053305 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:54:39.053380 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:54:39.053728 (+ 348us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:54:39.053730 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:54:39.053732 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:54:39.053999 (+ 267us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:54:39.054090 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0811 02:54:39.054854 (+ 764us) server_negotiation.cc:299] Negotiation successful
0811 02:54:39.054983 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":348}
W20260811 02:54:39.055528 28796 catalog_manager.cc:1558] T 00000000000000000000000000000000 P e912c1f4612b4e2aab6e32e1f1bf33f0: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 02:54:39.055624 28796 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 02:54:39.059415 28690 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:54:39.044972 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57044 (local address 127.12.138.253:40553)
0811 02:54:39.045074 (+ 102us) server_negotiation.cc:206] Beginning negotiation
0811 02:54:39.045078 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 02:54:39.045345 (+ 267us) server_negotiation.cc:407] Connection header received
0811 02:54:39.045496 (+ 151us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:54:39.045499 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:54:39.045542 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:54:39.045628 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:54:39.046229 (+ 601us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:54:39.046711 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:54:39.047535 (+ 824us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:54:39.047752 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:54:39.047809 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:54:39.052955 (+ 5146us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:54:39.052958 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:54:39.052961 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:54:39.052986 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:54:39.057362 (+ 4376us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:54:39.058000 (+ 638us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:54:39.058003 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:54:39.058005 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:54:39.058091 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:54:39.058461 (+ 370us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:54:39.058464 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:54:39.058466 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:54:39.058712 (+ 246us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:54:39.058895 (+ 183us) server_negotiation.cc:1035] Waiting for connection context
0811 02:54:39.059150 (+ 255us) server_negotiation.cc:299] Negotiation successful
0811 02:54:39.059274 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260811 02:54:39.060122 28775 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3fbdf7973fac4c11895279823cfcd3cf: Bootstrap starting.
I20260811 02:54:39.063056 28775 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3fbdf7973fac4c11895279823cfcd3cf: Neither blocks nor log segments found. Creating new log.
I20260811 02:54:39.053731 28606 master_runner.cc:186] Error getting master registration for 127.12.138.254:38245: OK, instance_id { permanent_uuid: "8a4b4a75a292410bb861f29bb3e7837b" instance_seqno: 1786416878520260 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260811 02:54:39.064010 28606 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260811 02:54:39.064030 28775 log.cc:824] T 00000000000000000000000000000000 P 3fbdf7973fac4c11895279823cfcd3cf: Log is configured to *not* fsync() on all Append() calls
I20260811 02:54:39.065411 12843 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 02:54:39.067510 28775 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3fbdf7973fac4c11895279823cfcd3cf: No bootstrap required, opened a new log
I20260811 02:54:39.071935 28775 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3fbdf7973fac4c11895279823cfcd3cf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a4b4a75a292410bb861f29bb3e7837b" member_type: VOTER last_known_addr { host: "127.12.138.254" port: 38245 } } peers { permanent_uuid: "e912c1f4612b4e2aab6e32e1f1bf33f0" member_type: VOTER last_known_addr { host: "127.12.138.253" port: 40553 } } peers { permanent_uuid: "3fbdf7973fac4c11895279823cfcd3cf" member_type: VOTER last_known_addr { host: "127.12.138.252" port: 45859 } }
I20260811 02:54:39.072230 28775 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3fbdf7973fac4c11895279823cfcd3cf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 02:54:39.072306 28775 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3fbdf7973fac4c11895279823cfcd3cf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3fbdf7973fac4c11895279823cfcd3cf, State: Initialized, Role: FOLLOWER
I20260811 02:54:39.072818 28775 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3fbdf7973fac4c11895279823cfcd3cf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a4b4a75a292410bb861f29bb3e7837b" member_type: VOTER last_known_addr { host: "127.12.138.254" port: 38245 } } peers { permanent_uuid: "e912c1f4612b4e2aab6e32e1f1bf33f0" member_type: VOTER last_known_addr { host: "127.12.138.253" port: 40553 } } peers { permanent_uuid: "3fbdf7973fac4c11895279823cfcd3cf" member_type: VOTER last_known_addr { host: "127.12.138.252" port: 45859 } }
I20260811 02:54:39.074368 28775 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3fbdf7973fac4c11895279823cfcd3cf [sys.catalog]: configured and running, proceeding with master startup.
I20260811 02:54:39.074570 28809 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3fbdf7973fac4c11895279823cfcd3cf [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a4b4a75a292410bb861f29bb3e7837b" member_type: VOTER last_known_addr { host: "127.12.138.254" port: 38245 } } peers { permanent_uuid: "e912c1f4612b4e2aab6e32e1f1bf33f0" member_type: VOTER last_known_addr { host: "127.12.138.253" port: 40553 } } peers { permanent_uuid: "3fbdf7973fac4c11895279823cfcd3cf" member_type: VOTER last_known_addr { host: "127.12.138.252" port: 45859 } } }
I20260811 02:54:39.074740 28809 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3fbdf7973fac4c11895279823cfcd3cf [sys.catalog]: This master's current role is: FOLLOWER
I20260811 02:54:39.078742 28690 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:54:39.068086 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57058 (local address 127.12.138.253:40553)
0811 02:54:39.068225 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0811 02:54:39.068229 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 02:54:39.069014 (+ 785us) server_negotiation.cc:407] Connection header received
0811 02:54:39.069157 (+ 143us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:54:39.069161 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:54:39.069212 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:54:39.069301 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:54:39.070331 (+ 1030us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:54:39.071102 (+ 771us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:54:39.072268 (+ 1166us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:54:39.072597 (+ 329us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:54:39.072706 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:54:39.074563 (+ 1857us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:54:39.074567 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:54:39.074570 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:54:39.074597 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:54:39.076518 (+ 1921us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:54:39.077025 (+ 507us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:54:39.077028 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:54:39.077030 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:54:39.077160 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:54:39.077463 (+ 303us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:54:39.077467 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:54:39.077470 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:54:39.077701 (+ 231us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:54:39.077834 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0811 02:54:39.078451 (+ 617us) server_negotiation.cc:299] Negotiation successful
0811 02:54:39.078567 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260811 02:54:39.081203 28603 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:54:39.067980 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35672 (local address 127.12.138.254:38245)
0811 02:54:39.068115 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0811 02:54:39.068120 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 02:54:39.068631 (+ 511us) server_negotiation.cc:407] Connection header received
0811 02:54:39.068852 (+ 221us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:54:39.068856 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:54:39.068914 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:54:39.068978 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:54:39.070400 (+ 1422us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:54:39.071227 (+ 827us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:54:39.072469 (+ 1242us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:54:39.072815 (+ 346us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:54:39.072903 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:54:39.076253 (+ 3350us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:54:39.076257 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:54:39.076260 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:54:39.076309 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:54:39.078405 (+ 2096us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:54:39.079483 (+ 1078us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:54:39.079488 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:54:39.079490 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:54:39.079583 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:54:39.079911 (+ 328us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:54:39.079915 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:54:39.079918 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:54:39.080171 (+ 253us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:54:39.080306 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0811 02:54:39.080773 (+ 467us) server_negotiation.cc:299] Negotiation successful
0811 02:54:39.080931 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260811 02:54:39.079572 28696 master_runner.cc:186] Error getting master registration for 127.12.138.254:38245: OK, instance_id { permanent_uuid: "8a4b4a75a292410bb861f29bb3e7837b" instance_seqno: 1786416878520260 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260811 02:54:39.082793 28696 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260811 02:54:39.091053 28781 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:54:39.068427 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37248 (local address 127.12.138.252:45859)
0811 02:54:39.068536 (+ 109us) server_negotiation.cc:206] Beginning negotiation
0811 02:54:39.068540 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 02:54:39.077475 (+ 8935us) server_negotiation.cc:407] Connection header received
0811 02:54:39.081817 (+ 4342us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:54:39.081822 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:54:39.081875 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:54:39.081959 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:54:39.082898 (+ 939us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:54:39.083685 (+ 787us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:54:39.084836 (+ 1151us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:54:39.085207 (+ 371us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:54:39.085286 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:54:39.087191 (+ 1905us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:54:39.087195 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:54:39.087198 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:54:39.087226 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:54:39.089041 (+ 1815us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:54:39.089584 (+ 543us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:54:39.089588 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:54:39.089590 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:54:39.089687 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:54:39.090001 (+ 314us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:54:39.090005 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:54:39.090007 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:54:39.090229 (+ 222us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:54:39.090361 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0811 02:54:39.090740 (+ 379us) server_negotiation.cc:299] Negotiation successful
0811 02:54:39.090875 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
W20260811 02:54:39.092882 28820 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 3fbdf7973fac4c11895279823cfcd3cf: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 02:54:39.092965 28820 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 02:54:39.161646 28599 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.12.138.252" port: 45859 } attempt: 4
Aug 11 02:54:39 dist-test-slave-9gcw krb5kdc[28504](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416878, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.138.254@KRBTEST.COM for kudu/127.12.138.252@KRBTEST.COM
I20260811 02:54:39.171837 28602 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:54:39.162108 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.138.252:45859 (local address 127.0.0.1:37260)
0811 02:54:39.162292 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0811 02:54:39.162307 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 02:54:39.162403 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:54:39.162678 (+ 275us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:54:39.162681 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:54:39.162913 (+ 232us) client_negotiation.cc:189] Negotiated authn=SASL
0811 02:54:39.163033 (+ 120us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:54:39.163043 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:54:39.164068 (+ 1025us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:54:39.164072 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:54:39.164944 (+ 872us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:54:39.164955 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:54:39.165552 (+ 597us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:54:39.165556 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:54:39.165666 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:54:39.166246 (+ 580us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 02:54:39.166262 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0811 02:54:39.167959 (+ 1697us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 02:54:39.170054 (+ 2095us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:54:39.170058 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:54:39.170060 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:54:39.170317 (+ 257us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:54:39.170658 (+ 341us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:54:39.170662 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:54:39.170664 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:54:39.170767 (+ 103us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:54:39.171253 (+ 486us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 02:54:39.171257 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 02:54:39.171367 (+ 110us) client_negotiation.cc:769] Sending connection context
0811 02:54:39.171480 (+ 113us) client_negotiation.cc:240] Negotiation successful
0811 02:54:39.171646 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":102}
I20260811 02:54:39.171904 28781 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:54:39.162193 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37260 (local address 127.12.138.252:45859)
0811 02:54:39.162315 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0811 02:54:39.162319 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 02:54:39.162363 (+ 44us) server_negotiation.cc:407] Connection header received
0811 02:54:39.162500 (+ 137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:54:39.162503 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:54:39.162551 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:54:39.162646 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:54:39.163160 (+ 514us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:54:39.163950 (+ 790us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:54:39.165070 (+ 1120us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:54:39.165430 (+ 360us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:54:39.165550 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:54:39.168104 (+ 2554us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:54:39.168108 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:54:39.168111 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:54:39.168141 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:54:39.169937 (+ 1796us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:54:39.170428 (+ 491us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:54:39.170432 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:54:39.170434 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:54:39.170543 (+ 109us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:54:39.170880 (+ 337us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:54:39.170884 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:54:39.170886 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:54:39.171149 (+ 263us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:54:39.171251 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0811 02:54:39.171562 (+ 311us) server_negotiation.cc:299] Negotiation successful
0811 02:54:39.171705 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260811 02:54:39.175803 28599 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8a4b4a75a292410bb861f29bb3e7837b: Bootstrap starting.
I20260811 02:54:39.179622 28599 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8a4b4a75a292410bb861f29bb3e7837b: Neither blocks nor log segments found. Creating new log.
I20260811 02:54:39.180540 28599 log.cc:824] T 00000000000000000000000000000000 P 8a4b4a75a292410bb861f29bb3e7837b: Log is configured to *not* fsync() on all Append() calls
I20260811 02:54:39.182838 28599 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8a4b4a75a292410bb861f29bb3e7837b: No bootstrap required, opened a new log
I20260811 02:54:39.186896 28599 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8a4b4a75a292410bb861f29bb3e7837b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a4b4a75a292410bb861f29bb3e7837b" member_type: VOTER last_known_addr { host: "127.12.138.254" port: 38245 } } peers { permanent_uuid: "e912c1f4612b4e2aab6e32e1f1bf33f0" member_type: VOTER last_known_addr { host: "127.12.138.253" port: 40553 } } peers { permanent_uuid: "3fbdf7973fac4c11895279823cfcd3cf" member_type: VOTER last_known_addr { host: "127.12.138.252" port: 45859 } }
I20260811 02:54:39.187139 28599 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8a4b4a75a292410bb861f29bb3e7837b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 02:54:39.187191 28599 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8a4b4a75a292410bb861f29bb3e7837b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8a4b4a75a292410bb861f29bb3e7837b, State: Initialized, Role: FOLLOWER
I20260811 02:54:39.187647 28599 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8a4b4a75a292410bb861f29bb3e7837b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a4b4a75a292410bb861f29bb3e7837b" member_type: VOTER last_known_addr { host: "127.12.138.254" port: 38245 } } peers { permanent_uuid: "e912c1f4612b4e2aab6e32e1f1bf33f0" member_type: VOTER last_known_addr { host: "127.12.138.253" port: 40553 } } peers { permanent_uuid: "3fbdf7973fac4c11895279823cfcd3cf" member_type: VOTER last_known_addr { host: "127.12.138.252" port: 45859 } }
I20260811 02:54:39.188201 28823 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8a4b4a75a292410bb861f29bb3e7837b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a4b4a75a292410bb861f29bb3e7837b" member_type: VOTER last_known_addr { host: "127.12.138.254" port: 38245 } } peers { permanent_uuid: "e912c1f4612b4e2aab6e32e1f1bf33f0" member_type: VOTER last_known_addr { host: "127.12.138.253" port: 40553 } } peers { permanent_uuid: "3fbdf7973fac4c11895279823cfcd3cf" member_type: VOTER last_known_addr { host: "127.12.138.252" port: 45859 } } }
I20260811 02:54:39.188405 28823 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8a4b4a75a292410bb861f29bb3e7837b [sys.catalog]: This master's current role is: FOLLOWER
I20260811 02:54:39.188980 28599 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8a4b4a75a292410bb861f29bb3e7837b [sys.catalog]: configured and running, proceeding with master startup.
W20260811 02:54:39.195349 28834 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 8a4b4a75a292410bb861f29bb3e7837b: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 02:54:39.195492 28834 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 02:54:39.198668 28517 master_runner.cc:263] No leader master found from master 8a4b4a75a292410bb861f29bb3e7837b
I20260811 02:54:39.198750 28517 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260811 02:54:39.243867 28726 tablet.cc:2369] T 00000000000000000000000000000000 P 3fbdf7973fac4c11895279823cfcd3cf: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 02:54:39.253326 28632 tablet.cc:2369] T 00000000000000000000000000000000 P e912c1f4612b4e2aab6e32e1f1bf33f0: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 02:54:39.266930 28544 tablet.cc:2369] T 00000000000000000000000000000000 P 8a4b4a75a292410bb861f29bb3e7837b: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 02:54:39.324249 28823 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8a4b4a75a292410bb861f29bb3e7837b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 02:54:39.324520 28823 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8a4b4a75a292410bb861f29bb3e7837b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a4b4a75a292410bb861f29bb3e7837b" member_type: VOTER last_known_addr { host: "127.12.138.254" port: 38245 } } peers { permanent_uuid: "e912c1f4612b4e2aab6e32e1f1bf33f0" member_type: VOTER last_known_addr { host: "127.12.138.253" port: 40553 } } peers { permanent_uuid: "3fbdf7973fac4c11895279823cfcd3cf" member_type: VOTER last_known_addr { host: "127.12.138.252" port: 45859 } }
I20260811 02:54:39.325784 28823 leader_election.cc:288] T 00000000000000000000000000000000 P 8a4b4a75a292410bb861f29bb3e7837b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e912c1f4612b4e2aab6e32e1f1bf33f0 (127.12.138.253:40553), 3fbdf7973fac4c11895279823cfcd3cf (127.12.138.252:45859)
I20260811 02:54:39.326354 28748 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "8a4b4a75a292410bb861f29bb3e7837b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3fbdf7973fac4c11895279823cfcd3cf" is_pre_election: true
I20260811 02:54:39.326584 28662 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "8a4b4a75a292410bb861f29bb3e7837b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e912c1f4612b4e2aab6e32e1f1bf33f0" is_pre_election: true
I20260811 02:54:39.326699 28748 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 3fbdf7973fac4c11895279823cfcd3cf [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a4b4a75a292410bb861f29bb3e7837b in term 0.
I20260811 02:54:39.326932 28662 raft_consensus.cc:2454] T 00000000000000000000000000000000 P e912c1f4612b4e2aab6e32e1f1bf33f0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a4b4a75a292410bb861f29bb3e7837b in term 0.
I20260811 02:54:39.327191 28541 leader_election.cc:302] T 00000000000000000000000000000000 P 8a4b4a75a292410bb861f29bb3e7837b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3fbdf7973fac4c11895279823cfcd3cf, 8a4b4a75a292410bb861f29bb3e7837b; no voters:
I20260811 02:54:39.327503 28823 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8a4b4a75a292410bb861f29bb3e7837b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 02:54:39.327598 28823 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8a4b4a75a292410bb861f29bb3e7837b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 02:54:39.327643 28823 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8a4b4a75a292410bb861f29bb3e7837b [term 0 FOLLOWER]: Advancing to term 1
I20260811 02:54:39.328748 28823 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8a4b4a75a292410bb861f29bb3e7837b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a4b4a75a292410bb861f29bb3e7837b" member_type: VOTER last_known_addr { host: "127.12.138.254" port: 38245 } } peers { permanent_uuid: "e912c1f4612b4e2aab6e32e1f1bf33f0" member_type: VOTER last_known_addr { host: "127.12.138.253" port: 40553 } } peers { permanent_uuid: "3fbdf7973fac4c11895279823cfcd3cf" member_type: VOTER last_known_addr { host: "127.12.138.252" port: 45859 } }
I20260811 02:54:39.329106 28823 leader_election.cc:288] T 00000000000000000000000000000000 P 8a4b4a75a292410bb861f29bb3e7837b [CANDIDATE]: Term 1 election: Requested vote from peers e912c1f4612b4e2aab6e32e1f1bf33f0 (127.12.138.253:40553), 3fbdf7973fac4c11895279823cfcd3cf (127.12.138.252:45859)
I20260811 02:54:39.329483 28748 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "8a4b4a75a292410bb861f29bb3e7837b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3fbdf7973fac4c11895279823cfcd3cf"
I20260811 02:54:39.329634 28748 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3fbdf7973fac4c11895279823cfcd3cf [term 0 FOLLOWER]: Advancing to term 1
I20260811 02:54:39.329738 28662 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "8a4b4a75a292410bb861f29bb3e7837b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e912c1f4612b4e2aab6e32e1f1bf33f0"
I20260811 02:54:39.329905 28662 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e912c1f4612b4e2aab6e32e1f1bf33f0 [term 0 FOLLOWER]: Advancing to term 1
I20260811 02:54:39.330772 28748 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 3fbdf7973fac4c11895279823cfcd3cf [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a4b4a75a292410bb861f29bb3e7837b in term 1.
I20260811 02:54:39.331069 28541 leader_election.cc:302] T 00000000000000000000000000000000 P 8a4b4a75a292410bb861f29bb3e7837b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3fbdf7973fac4c11895279823cfcd3cf, 8a4b4a75a292410bb861f29bb3e7837b; no voters:
I20260811 02:54:39.331238 28823 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8a4b4a75a292410bb861f29bb3e7837b [term 1 FOLLOWER]: Leader election won for term 1
I20260811 02:54:39.331368 28662 raft_consensus.cc:2454] T 00000000000000000000000000000000 P e912c1f4612b4e2aab6e32e1f1bf33f0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a4b4a75a292410bb861f29bb3e7837b in term 1.
I20260811 02:54:39.331528 28823 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8a4b4a75a292410bb861f29bb3e7837b [term 1 LEADER]: Becoming Leader. State: Replica: 8a4b4a75a292410bb861f29bb3e7837b, State: Running, Role: LEADER
I20260811 02:54:39.331858 28823 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8a4b4a75a292410bb861f29bb3e7837b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a4b4a75a292410bb861f29bb3e7837b" member_type: VOTER last_known_addr { host: "127.12.138.254" port: 38245 } } peers { permanent_uuid: "e912c1f4612b4e2aab6e32e1f1bf33f0" member_type: VOTER last_known_addr { host: "127.12.138.253" port: 40553 } } peers { permanent_uuid: "3fbdf7973fac4c11895279823cfcd3cf" member_type: VOTER last_known_addr { host: "127.12.138.252" port: 45859 } }
I20260811 02:54:39.335186 28836 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8a4b4a75a292410bb861f29bb3e7837b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8a4b4a75a292410bb861f29bb3e7837b. Latest consensus state: current_term: 1 leader_uuid: "8a4b4a75a292410bb861f29bb3e7837b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a4b4a75a292410bb861f29bb3e7837b" member_type: VOTER last_known_addr { host: "127.12.138.254" port: 38245 } } peers { permanent_uuid: "e912c1f4612b4e2aab6e32e1f1bf33f0" member_type: VOTER last_known_addr { host: "127.12.138.253" port: 40553 } } peers { permanent_uuid: "3fbdf7973fac4c11895279823cfcd3cf" member_type: VOTER last_known_addr { host: "127.12.138.252" port: 45859 } } }
I20260811 02:54:39.335299 28836 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8a4b4a75a292410bb861f29bb3e7837b [sys.catalog]: This master's current role is: LEADER
I20260811 02:54:39.335685 28839 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 02:54:39.336812 28839 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 02:54:39.343089 28748 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 3fbdf7973fac4c11895279823cfcd3cf [term 1 FOLLOWER]: Refusing update from remote peer 8a4b4a75a292410bb861f29bb3e7837b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 02:54:39.344012 28662 raft_consensus.cc:1261] T 00000000000000000000000000000000 P e912c1f4612b4e2aab6e32e1f1bf33f0 [term 1 FOLLOWER]: Refusing update from remote peer 8a4b4a75a292410bb861f29bb3e7837b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 02:54:39.344604 28836 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 8a4b4a75a292410bb861f29bb3e7837b [LEADER]: Connected to new peer: Peer: permanent_uuid: "3fbdf7973fac4c11895279823cfcd3cf" member_type: VOTER last_known_addr { host: "127.12.138.252" port: 45859 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 02:54:39.344888 28823 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 8a4b4a75a292410bb861f29bb3e7837b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e912c1f4612b4e2aab6e32e1f1bf33f0" member_type: VOTER last_known_addr { host: "127.12.138.253" port: 40553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 02:54:39.348713 28809 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3fbdf7973fac4c11895279823cfcd3cf [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8a4b4a75a292410bb861f29bb3e7837b. Latest consensus state: current_term: 1 leader_uuid: "8a4b4a75a292410bb861f29bb3e7837b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a4b4a75a292410bb861f29bb3e7837b" member_type: VOTER last_known_addr { host: "127.12.138.254" port: 38245 } } peers { permanent_uuid: "e912c1f4612b4e2aab6e32e1f1bf33f0" member_type: VOTER last_known_addr { host: "127.12.138.253" port: 40553 } } peers { permanent_uuid: "3fbdf7973fac4c11895279823cfcd3cf" member_type: VOTER last_known_addr { host: "127.12.138.252" port: 45859 } } }
I20260811 02:54:39.348867 28809 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3fbdf7973fac4c11895279823cfcd3cf [sys.catalog]: This master's current role is: FOLLOWER
I20260811 02:54:39.349135 28783 sys_catalog.cc:455] T 00000000000000000000000000000000 P e912c1f4612b4e2aab6e32e1f1bf33f0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8a4b4a75a292410bb861f29bb3e7837b. Latest consensus state: current_term: 1 leader_uuid: "8a4b4a75a292410bb861f29bb3e7837b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a4b4a75a292410bb861f29bb3e7837b" member_type: VOTER last_known_addr { host: "127.12.138.254" port: 38245 } } peers { permanent_uuid: "e912c1f4612b4e2aab6e32e1f1bf33f0" member_type: VOTER last_known_addr { host: "127.12.138.253" port: 40553 } } peers { permanent_uuid: "3fbdf7973fac4c11895279823cfcd3cf" member_type: VOTER last_known_addr { host: "127.12.138.252" port: 45859 } } }
I20260811 02:54:39.349320 28783 sys_catalog.cc:458] T 00000000000000000000000000000000 P e912c1f4612b4e2aab6e32e1f1bf33f0 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 02:54:39.351961 28823 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8a4b4a75a292410bb861f29bb3e7837b [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "8a4b4a75a292410bb861f29bb3e7837b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a4b4a75a292410bb861f29bb3e7837b" member_type: VOTER last_known_addr { host: "127.12.138.254" port: 38245 } } peers { permanent_uuid: "e912c1f4612b4e2aab6e32e1f1bf33f0" member_type: VOTER last_known_addr { host: "127.12.138.253" port: 40553 } } peers { permanent_uuid: "3fbdf7973fac4c11895279823cfcd3cf" member_type: VOTER last_known_addr { host: "127.12.138.252" port: 45859 } } }
I20260811 02:54:39.352873 28841 mvcc.cc:205] Tried to move back new op lower bound from 7317163537780367360 to 7317163537745719296. Current Snapshot: MvccSnapshot[applied={T|T < 7317163537780367360}]
I20260811 02:54:39.353037 28809 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3fbdf7973fac4c11895279823cfcd3cf [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "8a4b4a75a292410bb861f29bb3e7837b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a4b4a75a292410bb861f29bb3e7837b" member_type: VOTER last_known_addr { host: "127.12.138.254" port: 38245 } } peers { permanent_uuid: "e912c1f4612b4e2aab6e32e1f1bf33f0" member_type: VOTER last_known_addr { host: "127.12.138.253" port: 40553 } } peers { permanent_uuid: "3fbdf7973fac4c11895279823cfcd3cf" member_type: VOTER last_known_addr { host: "127.12.138.252" port: 45859 } } }
I20260811 02:54:39.353168 28809 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3fbdf7973fac4c11895279823cfcd3cf [sys.catalog]: This master's current role is: FOLLOWER
I20260811 02:54:39.353915 28783 sys_catalog.cc:455] T 00000000000000000000000000000000 P e912c1f4612b4e2aab6e32e1f1bf33f0 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "8a4b4a75a292410bb861f29bb3e7837b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a4b4a75a292410bb861f29bb3e7837b" member_type: VOTER last_known_addr { host: "127.12.138.254" port: 38245 } } peers { permanent_uuid: "e912c1f4612b4e2aab6e32e1f1bf33f0" member_type: VOTER last_known_addr { host: "127.12.138.253" port: 40553 } } peers { permanent_uuid: "3fbdf7973fac4c11895279823cfcd3cf" member_type: VOTER last_known_addr { host: "127.12.138.252" port: 45859 } } }
I20260811 02:54:39.354028 28783 sys_catalog.cc:458] T 00000000000000000000000000000000 P e912c1f4612b4e2aab6e32e1f1bf33f0 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 02:54:39.352334 28836 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8a4b4a75a292410bb861f29bb3e7837b [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "8a4b4a75a292410bb861f29bb3e7837b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a4b4a75a292410bb861f29bb3e7837b" member_type: VOTER last_known_addr { host: "127.12.138.254" port: 38245 } } peers { permanent_uuid: "e912c1f4612b4e2aab6e32e1f1bf33f0" member_type: VOTER last_known_addr { host: "127.12.138.253" port: 40553 } } peers { permanent_uuid: "3fbdf7973fac4c11895279823cfcd3cf" member_type: VOTER last_known_addr { host: "127.12.138.252" port: 45859 } } }
I20260811 02:54:39.353353 28839 catalog_manager.cc:1347] Generated new cluster ID: 8b25a4a47a1847f09dc57feffcb18643
I20260811 02:54:39.354626 28836 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8a4b4a75a292410bb861f29bb3e7837b [sys.catalog]: This master's current role is: LEADER
I20260811 02:54:39.354645 28839 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 02:54:39.354818 28823 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8a4b4a75a292410bb861f29bb3e7837b [sys.catalog]: This master's current role is: LEADER
I20260811 02:54:39.370568 28839 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 02:54:39.371970 28839 catalog_manager.cc:1504] Loading token signing keys...
I20260811 02:54:39.379009 28839 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8a4b4a75a292410bb861f29bb3e7837b: Generated new TSK 0
I20260811 02:54:39.380071 28839 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 02:54:40.057098 28796 catalog_manager.cc:1259] Loaded cluster ID: 8b25a4a47a1847f09dc57feffcb18643
I20260811 02:54:40.057237 28796 catalog_manager.cc:1552] T 00000000000000000000000000000000 P e912c1f4612b4e2aab6e32e1f1bf33f0: loading cluster ID for follower catalog manager: success
I20260811 02:54:40.059042 28796 catalog_manager.cc:1574] T 00000000000000000000000000000000 P e912c1f4612b4e2aab6e32e1f1bf33f0: acquiring CA information for follower catalog manager: success
I20260811 02:54:40.060298 28796 catalog_manager.cc:1602] T 00000000000000000000000000000000 P e912c1f4612b4e2aab6e32e1f1bf33f0: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260811 02:54:40.094349 28820 catalog_manager.cc:1259] Loaded cluster ID: 8b25a4a47a1847f09dc57feffcb18643
I20260811 02:54:40.094457 28820 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 3fbdf7973fac4c11895279823cfcd3cf: loading cluster ID for follower catalog manager: success
I20260811 02:54:40.096103 28820 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 3fbdf7973fac4c11895279823cfcd3cf: acquiring CA information for follower catalog manager: success
I20260811 02:54:40.097622 28820 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 3fbdf7973fac4c11895279823cfcd3cf: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260811 02:54:40.448372 12843 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDawEwS/build/debug/bin/kudu with pid 28517
I20260811 02:54:40.459995 12843 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDawEwS/build/debug/bin/kudu with pid 28606
I20260811 02:54:40.468168 12843 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDawEwS/build/debug/bin/kudu with pid 28696
2026-08-11T02:54:40Z chronyd exiting
[ OK ] SecurityITest.IPKICACert (3433 ms)
[ RUN ] SecurityITest.IPKICACertWebServerEndPoints
Loading random data
Initializing database '/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786416707705494-12843-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 02:54:40 dist-test-slave-9gcw krb5kdc[28856](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 02:54:40 dist-test-slave-9gcw krb5kdc[28856](info): set up 2 sockets
Aug 11 02:54:40 dist-test-slave-9gcw krb5kdc[28856](info): commencing operation
krb5kdc: starting...
W20260811 02:54:41.651490 12843 mini_kdc.cc:121] Time spent starting KDC: real 1.158s user 0.007s sys 0.001s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786416707705494-12843-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786416707705494-12843-0/krb5kdc/test-user.keytab.
Aug 11 02:54:41 dist-test-slave-9gcw krb5kdc[28856](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416881, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T02:54:41Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T02:54:41Z Disabled control of system clock
WARNING: no policy specified for kudu/127.12.138.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.138.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.138.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786416707705494-12843-0/krb5kdc/kudu_127.12.138.254.keytab.
Entry for principal kudu/127.12.138.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786416707705494-12843-0/krb5kdc/kudu_127.12.138.254.keytab.
I20260811 02:54:41.793996 12843 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDawEwS/build/debug/bin/kudu
/tmp/dist-test-taskDawEwS/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786416707705494-12843-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786416707705494-12843-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786416707705494-12843-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786416707705494-12843-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.138.254:43143
--webserver_interface=127.12.138.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786416707705494-12843-0/krb5kdc/kudu_127.12.138.254.keytab
--principal=kudu/127.12.138.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.138.212:35367
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.12.138.254:43143
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786416707705494-12843-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786416707705494-12843-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786416707705494-12843-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 02:54:41.914776 28869 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 02:54:41.915077 28869 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 02:54:41.915122 28869 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 02:54:41.919236 28869 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 02:54:41.919297 28869 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 02:54:41.919317 28869 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 02:54:41.919338 28869 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 02:54:41.919406 28869 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 02:54:41.925026 28869 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.138.212:35367
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786416707705494-12843-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786416707705494-12843-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.12.138.254:43143
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786416707705494-12843-0/krb5kdc/kudu_127.12.138.254.keytab
--principal=kudu/127.12.138.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.138.254:43143
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786416707705494-12843-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.12.138.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28869
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786416707705494-12843-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 02:09:49 UTC on ccycloud.quasar-fsnqqc.root.comops.site
build id 4579570
I20260811 02:54:41.926188 28869 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 02:54:41.927242 28869 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 02:54:41.936491 28878 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:54:41.936491 28875 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:54:41.936492 28876 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:54:41.937011 28869 server_base.cc:1034] running on GCE node
I20260811 02:54:41.937654 28869 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 02:54:41.938860 28869 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 02:54:41.940025 28869 hybrid_clock.cc:648] HybridClock initialized: now 1786416881939998 us; error 44 us; skew 500 ppm
Aug 11 02:54:41 dist-test-slave-9gcw krb5kdc[28856](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416881, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.138.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 02:54:41.942991 28869 init.cc:377] Logged in from keytab as kudu/127.12.138.254@KRBTEST.COM (short username kudu)
I20260811 02:54:41.943894 28869 webserver.cc:460] Webserver started at http://127.12.138.254:36749/ using document root <none> and password file <none>
I20260811 02:54:41.944442 28869 fs_manager.cc:359] Metadata directory not provided
I20260811 02:54:41.944501 28869 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 02:54:41.944686 28869 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 02:54:41.947037 28869 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786416707705494-12843-0/minicluster-data/master-0/data/instance:
uuid: "49ba02e688164e29b8a48d151f7e9d53"
format_stamp: "Formatted at 2026-08-11 02:54:41 on dist-test-slave-9gcw"
server_key: "dd3b521275cc213e5b0109b0b0444a58"
server_key_iv: "58938a4faa4abda575b81e66eebcb78b"
server_key_version: "encryptionkey@0"
I20260811 02:54:41.947580 28869 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786416707705494-12843-0/minicluster-data/master-0/wal/instance:
uuid: "49ba02e688164e29b8a48d151f7e9d53"
format_stamp: "Formatted at 2026-08-11 02:54:41 on dist-test-slave-9gcw"
server_key: "dd3b521275cc213e5b0109b0b0444a58"
server_key_iv: "58938a4faa4abda575b81e66eebcb78b"
server_key_version: "encryptionkey@0"
I20260811 02:54:41.951611 28869 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260811 02:54:41.954360 28886 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 02:54:41.955717 28869 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 02:54:41.955833 28869 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786416707705494-12843-0/minicluster-data/master-0/data,/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786416707705494-12843-0/minicluster-data/master-0/wal
uuid: "49ba02e688164e29b8a48d151f7e9d53"
format_stamp: "Formatted at 2026-08-11 02:54:41 on dist-test-slave-9gcw"
server_key: "dd3b521275cc213e5b0109b0b0444a58"
server_key_iv: "58938a4faa4abda575b81e66eebcb78b"
server_key_version: "encryptionkey@0"
I20260811 02:54:41.955938 28869 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786416707705494-12843-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786416707705494-12843-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786416707705494-12843-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 02:54:41.973052 28869 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 02:54:41.976444 28869 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 02:54:41.976627 28869 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 02:54:41.984376 28869 rpc_server.cc:307] RPC server started. Bound to: 127.12.138.254:43143
I20260811 02:54:41.984396 28952 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.138.254:43143 every 8 connection(s)
I20260811 02:54:41.985497 28869 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786416707705494-12843-0/minicluster-data/master-0/data/info.pb
I20260811 02:54:41.988454 28954 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 02:54:41.992141 12843 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDawEwS/build/debug/bin/kudu as pid 28869
I20260811 02:54:41.992305 12843 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786416707705494-12843-0/minicluster-data/master-0/wal/instance
I20260811 02:54:41.992607 12843 external_mini_cluster.cc:1428] Setting key f71178385fe60b14712b239a9a6e6072
I20260811 02:54:41.995023 28954 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 49ba02e688164e29b8a48d151f7e9d53: Bootstrap starting.
I20260811 02:54:41.997370 28954 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 49ba02e688164e29b8a48d151f7e9d53: Neither blocks nor log segments found. Creating new log.
I20260811 02:54:41.998157 28954 log.cc:824] T 00000000000000000000000000000000 P 49ba02e688164e29b8a48d151f7e9d53: Log is configured to *not* fsync() on all Append() calls
I20260811 02:54:42.000116 28954 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 49ba02e688164e29b8a48d151f7e9d53: No bootstrap required, opened a new log
Aug 11 02:54:42 dist-test-slave-9gcw krb5kdc[28856](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416881, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.12.138.254@KRBTEST.COM
I20260811 02:54:42.004629 28954 raft_consensus.cc:348] T 00000000000000000000000000000000 P 49ba02e688164e29b8a48d151f7e9d53 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "49ba02e688164e29b8a48d151f7e9d53" member_type: VOTER last_known_addr { host: "127.12.138.254" port: 43143 } }
I20260811 02:54:42.004938 28954 raft_consensus.cc:374] T 00000000000000000000000000000000 P 49ba02e688164e29b8a48d151f7e9d53 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 02:54:42.004981 28954 raft_consensus.cc:729] T 00000000000000000000000000000000 P 49ba02e688164e29b8a48d151f7e9d53 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 49ba02e688164e29b8a48d151f7e9d53, State: Initialized, Role: FOLLOWER
I20260811 02:54:42.005573 28954 consensus_queue.cc:260] T 00000000000000000000000000000000 P 49ba02e688164e29b8a48d151f7e9d53 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "49ba02e688164e29b8a48d151f7e9d53" member_type: VOTER last_known_addr { host: "127.12.138.254" port: 43143 } }
I20260811 02:54:42.005738 28954 raft_consensus.cc:388] T 00000000000000000000000000000000 P 49ba02e688164e29b8a48d151f7e9d53 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 02:54:42.005820 28954 raft_consensus.cc:482] T 00000000000000000000000000000000 P 49ba02e688164e29b8a48d151f7e9d53 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 02:54:42.005960 28954 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 49ba02e688164e29b8a48d151f7e9d53 [term 0 FOLLOWER]: Advancing to term 1
I20260811 02:54:42.006978 28954 raft_consensus.cc:504] T 00000000000000000000000000000000 P 49ba02e688164e29b8a48d151f7e9d53 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "49ba02e688164e29b8a48d151f7e9d53" member_type: VOTER last_known_addr { host: "127.12.138.254" port: 43143 } }
I20260811 02:54:42.007355 28954 leader_election.cc:302] T 00000000000000000000000000000000 P 49ba02e688164e29b8a48d151f7e9d53 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 49ba02e688164e29b8a48d151f7e9d53; no voters:
I20260811 02:54:42.007750 28954 leader_election.cc:288] T 00000000000000000000000000000000 P 49ba02e688164e29b8a48d151f7e9d53 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 02:54:42.007902 28959 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 49ba02e688164e29b8a48d151f7e9d53 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 02:54:42.008147 28959 raft_consensus.cc:686] T 00000000000000000000000000000000 P 49ba02e688164e29b8a48d151f7e9d53 [term 1 LEADER]: Becoming Leader. State: Replica: 49ba02e688164e29b8a48d151f7e9d53, State: Running, Role: LEADER
I20260811 02:54:42.008577 28959 consensus_queue.cc:237] T 00000000000000000000000000000000 P 49ba02e688164e29b8a48d151f7e9d53 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "49ba02e688164e29b8a48d151f7e9d53" member_type: VOTER last_known_addr { host: "127.12.138.254" port: 43143 } }
I20260811 02:54:42.009243 28954 sys_catalog.cc:564] T 00000000000000000000000000000000 P 49ba02e688164e29b8a48d151f7e9d53 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 02:54:42.009339 28957 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:54:41.994267 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34652 (local address 127.12.138.254:43143)
0811 02:54:41.994770 (+ 503us) server_negotiation.cc:206] Beginning negotiation
0811 02:54:41.994786 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0811 02:54:41.994824 (+ 38us) server_negotiation.cc:407] Connection header received
0811 02:54:41.995567 (+ 743us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:54:41.995587 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:54:41.996018 (+ 431us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:54:41.996393 (+ 375us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:54:41.997280 (+ 887us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:54:41.998424 (+ 1144us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:54:41.999337 (+ 913us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:54:41.999859 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:54:42.000055 (+ 196us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:54:42.003068 (+ 3013us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:54:42.003074 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:54:42.003090 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:54:42.003121 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:54:42.005555 (+ 2434us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:54:42.006250 (+ 695us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:54:42.006258 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:54:42.006264 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:54:42.006408 (+ 144us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:54:42.006751 (+ 343us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:54:42.006755 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:54:42.006757 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:54:42.007310 (+ 553us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:54:42.007559 (+ 249us) server_negotiation.cc:1035] Waiting for connection context
0811 02:54:42.007920 (+ 361us) server_negotiation.cc:299] Negotiation successful
0811 02:54:42.008323 (+ 403us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":336,"thread_start_us":151,"threads_started":1}
I20260811 02:54:42.009897 28961 sys_catalog.cc:455] T 00000000000000000000000000000000 P 49ba02e688164e29b8a48d151f7e9d53 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 49ba02e688164e29b8a48d151f7e9d53. Latest consensus state: current_term: 1 leader_uuid: "49ba02e688164e29b8a48d151f7e9d53" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "49ba02e688164e29b8a48d151f7e9d53" member_type: VOTER last_known_addr { host: "127.12.138.254" port: 43143 } } }
I20260811 02:54:42.010040 28961 sys_catalog.cc:458] T 00000000000000000000000000000000 P 49ba02e688164e29b8a48d151f7e9d53 [sys.catalog]: This master's current role is: LEADER
I20260811 02:54:42.010325 28960 sys_catalog.cc:455] T 00000000000000000000000000000000 P 49ba02e688164e29b8a48d151f7e9d53 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "49ba02e688164e29b8a48d151f7e9d53" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "49ba02e688164e29b8a48d151f7e9d53" member_type: VOTER last_known_addr { host: "127.12.138.254" port: 43143 } } }
I20260811 02:54:42.010429 28960 sys_catalog.cc:458] T 00000000000000000000000000000000 P 49ba02e688164e29b8a48d151f7e9d53 [sys.catalog]: This master's current role is: LEADER
I20260811 02:54:42.012877 12843 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
W20260811 02:54:42.015080 28976 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 49ba02e688164e29b8a48d151f7e9d53: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 02:54:42.015179 28976 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 02:54:42.015281 28966 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 02:54:42.016059 28966 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 02:54:42.022555 28966 catalog_manager.cc:1347] Generated new cluster ID: ef7b23b446cc433b969c478d2980ac97
I20260811 02:54:42.022657 28966 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 02:54:42.053349 28966 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 02:54:42.054757 28966 catalog_manager.cc:1504] Loading token signing keys...
I20260811 02:54:42.067178 28966 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 49ba02e688164e29b8a48d151f7e9d53: Generated new TSK 0
I20260811 02:54:42.068048 28966 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 02:54:42.120808 12843 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDawEwS/build/debug/bin/kudu with pid 28869
2026-08-11T02:54:42Z chronyd exiting
[ OK ] SecurityITest.IPKICACertWebServerEndPoints (1652 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedIPKICert
Loading random data
Initializing database '/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786416707705494-12843-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 02:54:42 dist-test-slave-9gcw krb5kdc[28987](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 02:54:42 dist-test-slave-9gcw krb5kdc[28987](info): set up 2 sockets
Aug 11 02:54:42 dist-test-slave-9gcw krb5kdc[28987](info): commencing operation
krb5kdc: starting...
W20260811 02:54:43.319572 12843 mini_kdc.cc:121] Time spent starting KDC: real 1.173s user 0.004s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786416707705494-12843-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786416707705494-12843-0/krb5kdc/test-user.keytab.
Aug 11 02:54:43 dist-test-slave-9gcw krb5kdc[28987](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416883, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T02:54:43Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T02:54:43Z Disabled control of system clock
WARNING: no policy specified for kudu/127.12.138.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.138.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.138.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786416707705494-12843-0/krb5kdc/kudu_127.12.138.254.keytab.
Entry for principal kudu/127.12.138.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786416707705494-12843-0/krb5kdc/kudu_127.12.138.254.keytab.
I20260811 02:54:43.457757 12843 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDawEwS/build/debug/bin/kudu
/tmp/dist-test-taskDawEwS/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786416707705494-12843-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786416707705494-12843-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786416707705494-12843-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786416707705494-12843-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.138.254:45409
--webserver_interface=127.12.138.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786416707705494-12843-0/krb5kdc/kudu_127.12.138.254.keytab
--principal=kudu/127.12.138.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.138.212:45835
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.12.138.254:45409
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786416707705494-12843-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786416707705494-12843-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786416707705494-12843-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 02:54:43.577354 29001 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 02:54:43.577685 29001 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 02:54:43.577766 29001 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 02:54:43.581846 29001 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 02:54:43.581954 29001 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 02:54:43.581990 29001 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 02:54:43.582028 29001 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 02:54:43.582062 29001 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 02:54:43.588593 29001 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.138.212:45835
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786416707705494-12843-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786416707705494-12843-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.12.138.254:45409
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786416707705494-12843-0/krb5kdc/kudu_127.12.138.254.keytab
--principal=kudu/127.12.138.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.138.254:45409
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786416707705494-12843-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.12.138.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29001
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786416707705494-12843-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 02:09:49 UTC on ccycloud.quasar-fsnqqc.root.comops.site
build id 4579570
I20260811 02:54:43.589870 29001 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 02:54:43.590997 29001 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 02:54:43.600083 29009 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:54:43.600106 29010 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:54:43.600132 29012 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:54:43.600348 29001 server_base.cc:1034] running on GCE node
I20260811 02:54:43.600862 29001 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 02:54:43.602073 29001 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 02:54:43.603245 29001 hybrid_clock.cc:648] HybridClock initialized: now 1786416883603229 us; error 28 us; skew 500 ppm
Aug 11 02:54:43 dist-test-slave-9gcw krb5kdc[28987](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416883, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.138.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 02:54:43.606295 29001 init.cc:377] Logged in from keytab as kudu/127.12.138.254@KRBTEST.COM (short username kudu)
I20260811 02:54:43.607239 29001 webserver.cc:460] Webserver started at http://127.12.138.254:46571/ using document root <none> and password file <none>
I20260811 02:54:43.607759 29001 fs_manager.cc:359] Metadata directory not provided
I20260811 02:54:43.607820 29001 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 02:54:43.608024 29001 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 02:54:43.610200 29001 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786416707705494-12843-0/minicluster-data/master-0/data/instance:
uuid: "11144008f58842b883c076177aaad052"
format_stamp: "Formatted at 2026-08-11 02:54:43 on dist-test-slave-9gcw"
server_key: "2ee95749916bbaa0295a7c0d9ba9a695"
server_key_iv: "f7e08cdfd20e96649c8e5c4ac6e77e5d"
server_key_version: "encryptionkey@0"
I20260811 02:54:43.610834 29001 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786416707705494-12843-0/minicluster-data/master-0/wal/instance:
uuid: "11144008f58842b883c076177aaad052"
format_stamp: "Formatted at 2026-08-11 02:54:43 on dist-test-slave-9gcw"
server_key: "2ee95749916bbaa0295a7c0d9ba9a695"
server_key_iv: "f7e08cdfd20e96649c8e5c4ac6e77e5d"
server_key_version: "encryptionkey@0"
I20260811 02:54:43.614637 29001 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260811 02:54:43.617184 29020 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 02:54:43.618446 29001 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260811 02:54:43.618590 29001 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786416707705494-12843-0/minicluster-data/master-0/data,/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786416707705494-12843-0/minicluster-data/master-0/wal
uuid: "11144008f58842b883c076177aaad052"
format_stamp: "Formatted at 2026-08-11 02:54:43 on dist-test-slave-9gcw"
server_key: "2ee95749916bbaa0295a7c0d9ba9a695"
server_key_iv: "f7e08cdfd20e96649c8e5c4ac6e77e5d"
server_key_version: "encryptionkey@0"
I20260811 02:54:43.618706 29001 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786416707705494-12843-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786416707705494-12843-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786416707705494-12843-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 02:54:43.632347 29001 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 02:54:43.635999 29001 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 02:54:43.636225 29001 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 02:54:43.644232 29001 rpc_server.cc:307] RPC server started. Bound to: 127.12.138.254:45409
I20260811 02:54:43.644837 29077 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.138.254:45409 every 8 connection(s)
I20260811 02:54:43.645617 29001 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786416707705494-12843-0/minicluster-data/master-0/data/info.pb
I20260811 02:54:43.646041 12843 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDawEwS/build/debug/bin/kudu as pid 29001
I20260811 02:54:43.646176 12843 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786416707705494-12843-0/minicluster-data/master-0/wal/instance
I20260811 02:54:43.646544 12843 external_mini_cluster.cc:1428] Setting key 04c37d63bb41908a03705627b1838cbf
I20260811 02:54:43.649214 29078 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 02:54:43.654886 29078 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 11144008f58842b883c076177aaad052: Bootstrap starting.
Aug 11 02:54:43 dist-test-slave-9gcw krb5kdc[28987](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416883, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.12.138.254@KRBTEST.COM
I20260811 02:54:43.657404 29078 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 11144008f58842b883c076177aaad052: Neither blocks nor log segments found. Creating new log.
I20260811 02:54:43.658159 29078 log.cc:824] T 00000000000000000000000000000000 P 11144008f58842b883c076177aaad052: Log is configured to *not* fsync() on all Append() calls
I20260811 02:54:43.660166 29078 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 11144008f58842b883c076177aaad052: No bootstrap required, opened a new log
I20260811 02:54:43.660917 29082 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:54:43.648041 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58624 (local address 127.12.138.254:45409)
0811 02:54:43.648490 (+ 449us) server_negotiation.cc:206] Beginning negotiation
0811 02:54:43.648506 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0811 02:54:43.648540 (+ 34us) server_negotiation.cc:407] Connection header received
0811 02:54:43.649449 (+ 909us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:54:43.649471 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:54:43.649800 (+ 329us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:54:43.650150 (+ 350us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:54:43.650971 (+ 821us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:54:43.652005 (+ 1034us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:54:43.652859 (+ 854us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:54:43.653405 (+ 546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:54:43.653571 (+ 166us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:54:43.656172 (+ 2601us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:54:43.656177 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:54:43.656189 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:54:43.656216 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:54:43.658049 (+ 1833us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:54:43.658539 (+ 490us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:54:43.658544 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:54:43.658548 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:54:43.658663 (+ 115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:54:43.658958 (+ 295us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:54:43.658962 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:54:43.658963 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:54:43.659367 (+ 404us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:54:43.659560 (+ 193us) server_negotiation.cc:1035] Waiting for connection context
0811 02:54:43.659865 (+ 305us) server_negotiation.cc:299] Negotiation successful
0811 02:54:43.660172 (+ 307us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":307,"thread_start_us":158,"threads_started":1}
I20260811 02:54:43.664233 29078 raft_consensus.cc:348] T 00000000000000000000000000000000 P 11144008f58842b883c076177aaad052 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "11144008f58842b883c076177aaad052" member_type: VOTER last_known_addr { host: "127.12.138.254" port: 45409 } }
I20260811 02:54:43.664476 29078 raft_consensus.cc:374] T 00000000000000000000000000000000 P 11144008f58842b883c076177aaad052 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 02:54:43.664554 29078 raft_consensus.cc:729] T 00000000000000000000000000000000 P 11144008f58842b883c076177aaad052 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 11144008f58842b883c076177aaad052, State: Initialized, Role: FOLLOWER
I20260811 02:54:43.665016 29078 consensus_queue.cc:260] T 00000000000000000000000000000000 P 11144008f58842b883c076177aaad052 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "11144008f58842b883c076177aaad052" member_type: VOTER last_known_addr { host: "127.12.138.254" port: 45409 } }
I20260811 02:54:43.665185 29078 raft_consensus.cc:388] T 00000000000000000000000000000000 P 11144008f58842b883c076177aaad052 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 02:54:43.665259 29078 raft_consensus.cc:482] T 00000000000000000000000000000000 P 11144008f58842b883c076177aaad052 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 02:54:43.665367 29078 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 11144008f58842b883c076177aaad052 [term 0 FOLLOWER]: Advancing to term 1
I20260811 02:54:43.666285 29078 raft_consensus.cc:504] T 00000000000000000000000000000000 P 11144008f58842b883c076177aaad052 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "11144008f58842b883c076177aaad052" member_type: VOTER last_known_addr { host: "127.12.138.254" port: 45409 } }
I20260811 02:54:43.666628 29078 leader_election.cc:302] T 00000000000000000000000000000000 P 11144008f58842b883c076177aaad052 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 11144008f58842b883c076177aaad052; no voters:
I20260811 02:54:43.666945 29078 leader_election.cc:288] T 00000000000000000000000000000000 P 11144008f58842b883c076177aaad052 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 02:54:43.667062 29084 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 11144008f58842b883c076177aaad052 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 02:54:43.667308 29084 raft_consensus.cc:686] T 00000000000000000000000000000000 P 11144008f58842b883c076177aaad052 [term 1 LEADER]: Becoming Leader. State: Replica: 11144008f58842b883c076177aaad052, State: Running, Role: LEADER
I20260811 02:54:43.667618 29084 consensus_queue.cc:237] T 00000000000000000000000000000000 P 11144008f58842b883c076177aaad052 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "11144008f58842b883c076177aaad052" member_type: VOTER last_known_addr { host: "127.12.138.254" port: 45409 } }
I20260811 02:54:43.668205 29078 sys_catalog.cc:564] T 00000000000000000000000000000000 P 11144008f58842b883c076177aaad052 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 02:54:43.669746 29085 sys_catalog.cc:455] T 00000000000000000000000000000000 P 11144008f58842b883c076177aaad052 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 11144008f58842b883c076177aaad052. Latest consensus state: current_term: 1 leader_uuid: "11144008f58842b883c076177aaad052" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "11144008f58842b883c076177aaad052" member_type: VOTER last_known_addr { host: "127.12.138.254" port: 45409 } } }
I20260811 02:54:43.669889 29085 sys_catalog.cc:458] T 00000000000000000000000000000000 P 11144008f58842b883c076177aaad052 [sys.catalog]: This master's current role is: LEADER
I20260811 02:54:43.670185 29087 sys_catalog.cc:455] T 00000000000000000000000000000000 P 11144008f58842b883c076177aaad052 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "11144008f58842b883c076177aaad052" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "11144008f58842b883c076177aaad052" member_type: VOTER last_known_addr { host: "127.12.138.254" port: 45409 } } }
I20260811 02:54:43.670301 29087 sys_catalog.cc:458] T 00000000000000000000000000000000 P 11144008f58842b883c076177aaad052 [sys.catalog]: This master's current role is: LEADER
I20260811 02:54:43.670538 29096 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 02:54:43.673331 29096 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 02:54:43.678378 29096 catalog_manager.cc:1347] Generated new cluster ID: d19c1050b4cd4f2e8db89c9e58687fed
I20260811 02:54:43.678449 29096 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 02:54:43.722877 29096 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 02:54:43.724248 29096 catalog_manager.cc:1504] Loading token signing keys...
I20260811 02:54:43.736063 29096 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 11144008f58842b883c076177aaad052: Generated new TSK 0
I20260811 02:54:43.736874 29096 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.12.138.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.138.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.138.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786416707705494-12843-0/krb5kdc/kudu_127.12.138.193.keytab.
Entry for principal kudu/127.12.138.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786416707705494-12843-0/krb5kdc/kudu_127.12.138.193.keytab.
I20260811 02:54:43.769676 12843 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDawEwS/build/debug/bin/kudu
/tmp/dist-test-taskDawEwS/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786416707705494-12843-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786416707705494-12843-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786416707705494-12843-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786416707705494-12843-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.138.193:0
--local_ip_for_outbound_sockets=127.12.138.193
--webserver_interface=127.12.138.193
--webserver_port=0
--tserver_master_addrs=127.12.138.254:45409
--keytab_file=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786416707705494-12843-0/krb5kdc/kudu_127.12.138.193.keytab
--principal=kudu/127.12.138.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.138.212:45835
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786416707705494-12843-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786416707705494-12843-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786416707705494-12843-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 02:54:43.887696 29109 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 02:54:43.887987 29109 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 02:54:43.888032 29109 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 02:54:43.892410 29109 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 02:54:43.892482 29109 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 02:54:43.892544 29109 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.138.193
I20260811 02:54:43.898185 29109 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.138.212:45835
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786416707705494-12843-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786416707705494-12843-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786416707705494-12843-0/krb5kdc/kudu_127.12.138.193.keytab
--principal=kudu/127.12.138.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.138.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786416707705494-12843-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.12.138.193
--webserver_port=0
--tserver_master_addrs=127.12.138.254:45409
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29109
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.138.193
--log_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786416707705494-12843-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 02:09:49 UTC on ccycloud.quasar-fsnqqc.root.comops.site
build id 4579570
I20260811 02:54:43.899320 29109 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 02:54:43.900377 29109 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 02:54:43.910516 29115 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:54:43.910553 29117 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:54:43.910514 29114 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:54:43.910861 29109 server_base.cc:1034] running on GCE node
I20260811 02:54:43.911377 29109 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 02:54:43.912056 29109 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 02:54:43.913223 29109 hybrid_clock.cc:648] HybridClock initialized: now 1786416883913181 us; error 52 us; skew 500 ppm
Aug 11 02:54:43 dist-test-slave-9gcw krb5kdc[28987](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416883, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.138.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 02:54:43.917279 29109 init.cc:377] Logged in from keytab as kudu/127.12.138.193@KRBTEST.COM (short username kudu)
I20260811 02:54:43.919106 29109 webserver.cc:460] Webserver started at http://127.12.138.193:34989/ using document root <none> and password file <none>
I20260811 02:54:43.919983 29109 fs_manager.cc:359] Metadata directory not provided
I20260811 02:54:43.920375 29109 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 02:54:43.920682 29109 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 02:54:43.925273 29109 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786416707705494-12843-0/minicluster-data/ts-0/data/instance:
uuid: "13a904a6ad6a46a78dd222ee22eaea3c"
format_stamp: "Formatted at 2026-08-11 02:54:43 on dist-test-slave-9gcw"
server_key: "03fc6e3876c6257fb7637fbb9ccc8ff4"
server_key_iv: "68781f4cd4ccd5db115f71fa3f511505"
server_key_version: "encryptionkey@0"
I20260811 02:54:43.926363 29109 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786416707705494-12843-0/minicluster-data/ts-0/wal/instance:
uuid: "13a904a6ad6a46a78dd222ee22eaea3c"
format_stamp: "Formatted at 2026-08-11 02:54:43 on dist-test-slave-9gcw"
server_key: "03fc6e3876c6257fb7637fbb9ccc8ff4"
server_key_iv: "68781f4cd4ccd5db115f71fa3f511505"
server_key_version: "encryptionkey@0"
I20260811 02:54:43.932415 29109 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.006s sys 0.000s
I20260811 02:54:43.936309 29125 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 02:54:43.938457 29109 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260811 02:54:43.938666 29109 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786416707705494-12843-0/minicluster-data/ts-0/data,/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786416707705494-12843-0/minicluster-data/ts-0/wal
uuid: "13a904a6ad6a46a78dd222ee22eaea3c"
format_stamp: "Formatted at 2026-08-11 02:54:43 on dist-test-slave-9gcw"
server_key: "03fc6e3876c6257fb7637fbb9ccc8ff4"
server_key_iv: "68781f4cd4ccd5db115f71fa3f511505"
server_key_version: "encryptionkey@0"
I20260811 02:54:43.938843 29109 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786416707705494-12843-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786416707705494-12843-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786416707705494-12843-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 02:54:43.986887 29109 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 02:54:43.990507 29109 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 02:54:43.990734 29109 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 02:54:43.991371 29109 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 02:54:43.992246 29109 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 02:54:43.992316 29109 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 02:54:43.992385 29109 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 02:54:43.992434 29109 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 02:54:44.001873 29109 rpc_server.cc:307] RPC server started. Bound to: 127.12.138.193:40085
I20260811 02:54:44.001956 29254 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.138.193:40085 every 8 connection(s)
I20260811 02:54:44.003328 29109 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786416707705494-12843-0/minicluster-data/ts-0/data/info.pb
I20260811 02:54:44.006798 12843 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDawEwS/build/debug/bin/kudu as pid 29109
I20260811 02:54:44.006949 12843 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786416707705494-12843-0/minicluster-data/ts-0/wal/instance
I20260811 02:54:44.007299 12843 external_mini_cluster.cc:1428] Setting key 29d644125cec0f559d495591b6e6a5de
Aug 11 02:54:44 dist-test-slave-9gcw krb5kdc[28987](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416883, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.138.193@KRBTEST.COM for kudu/127.12.138.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.12.138.194@KRBTEST.COM; defaulting to no policy
I20260811 02:54:44.022603 29082 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:54:44.005716 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.138.193:52951 (local address 127.12.138.254:45409)
0811 02:54:44.005867 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0811 02:54:44.005872 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 02:54:44.007021 (+ 1149us) server_negotiation.cc:407] Connection header received
0811 02:54:44.009437 (+ 2416us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:54:44.009440 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:54:44.009496 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:54:44.009589 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:54:44.010849 (+ 1260us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:54:44.011721 (+ 872us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:54:44.013815 (+ 2094us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:54:44.014306 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:54:44.014413 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:54:44.018283 (+ 3870us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:54:44.018290 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:54:44.018296 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:54:44.018338 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:54:44.020364 (+ 2026us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:54:44.020978 (+ 614us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:54:44.020980 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:54:44.020981 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:54:44.021059 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:54:44.021479 (+ 420us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:54:44.021483 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:54:44.021484 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:54:44.021727 (+ 243us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:54:44.021819 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0811 02:54:44.022356 (+ 537us) server_negotiation.cc:299] Negotiation successful
0811 02:54:44.022473 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.138.194@KRBTEST.COM" created.
I20260811 02:54:44.023784 29258 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:54:44.006261 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.138.254:45409 (local address 127.12.138.193:52951)
0811 02:54:44.006846 (+ 585us) negotiation.cc:107] Waiting for socket to connect
0811 02:54:44.006902 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0811 02:54:44.009167 (+ 2265us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:54:44.009774 (+ 607us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:54:44.009781 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:54:44.010168 (+ 387us) client_negotiation.cc:189] Negotiated authn=SASL
0811 02:54:44.010676 (+ 508us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:54:44.010690 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:54:44.011910 (+ 1220us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:54:44.011917 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:54:44.013649 (+ 1732us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:54:44.013662 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:54:44.014657 (+ 995us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:54:44.014663 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:54:44.014916 (+ 253us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:54:44.015702 (+ 786us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 02:54:44.015732 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0811 02:54:44.018049 (+ 2317us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 02:54:44.020490 (+ 2441us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:54:44.020500 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:54:44.020523 (+ 23us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:54:44.020862 (+ 339us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:54:44.021180 (+ 318us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:54:44.021186 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:54:44.021190 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:54:44.021375 (+ 185us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:54:44.021830 (+ 455us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 02:54:44.021835 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 02:54:44.022131 (+ 296us) client_negotiation.cc:769] Sending connection context
0811 02:54:44.022314 (+ 183us) client_negotiation.cc:240] Negotiation successful
0811 02:54:44.022652 (+ 338us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":335,"thread_start_us":167,"threads_started":1}
I20260811 02:54:44.025177 29255 heartbeater.cc:347] Connected to a master server at 127.12.138.254:45409
I20260811 02:54:44.025460 29255 heartbeater.cc:464] Registering TS with master...
I20260811 02:54:44.026026 29255 heartbeater.cc:511] Master 127.12.138.254:45409 requested a full tablet report, sending...
I20260811 02:54:44.027915 29039 ts_manager.cc:194] Registered new tserver with Master: 13a904a6ad6a46a78dd222ee22eaea3c (127.12.138.193:40085)
I20260811 02:54:44.029711 29039 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.12.138.193@KRBTEST.COM'} at 127.12.138.193:52951
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.138.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786416707705494-12843-0/krb5kdc/kudu_127.12.138.194.keytab.
Entry for principal kudu/127.12.138.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786416707705494-12843-0/krb5kdc/kudu_127.12.138.194.keytab.
I20260811 02:54:44.045744 12843 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDawEwS/build/debug/bin/kudu
/tmp/dist-test-taskDawEwS/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786416707705494-12843-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786416707705494-12843-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786416707705494-12843-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786416707705494-12843-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.138.194:0
--local_ip_for_outbound_sockets=127.12.138.194
--webserver_interface=127.12.138.194
--webserver_port=0
--tserver_master_addrs=127.12.138.254:45409
--keytab_file=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786416707705494-12843-0/krb5kdc/kudu_127.12.138.194.keytab
--principal=kudu/127.12.138.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.138.212:45835
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786416707705494-12843-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786416707705494-12843-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786416707705494-12843-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 02:54:44.164167 29261 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 02:54:44.164444 29261 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 02:54:44.164489 29261 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 02:54:44.168650 29261 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 02:54:44.168740 29261 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 02:54:44.168846 29261 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.138.194
I20260811 02:54:44.174469 29261 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.138.212:45835
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786416707705494-12843-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786416707705494-12843-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786416707705494-12843-0/krb5kdc/kudu_127.12.138.194.keytab
--principal=kudu/127.12.138.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.138.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786416707705494-12843-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.12.138.194
--webserver_port=0
--tserver_master_addrs=127.12.138.254:45409
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29261
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.138.194
--log_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786416707705494-12843-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 02:09:49 UTC on ccycloud.quasar-fsnqqc.root.comops.site
build id 4579570
I20260811 02:54:44.175624 29261 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 02:54:44.176657 29261 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 02:54:44.187872 29267 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:54:44.187896 29270 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:54:44.187896 29268 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:54:44.188149 29261 server_base.cc:1034] running on GCE node
I20260811 02:54:44.188611 29261 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 02:54:44.189375 29261 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 02:54:44.190568 29261 hybrid_clock.cc:648] HybridClock initialized: now 1786416884190554 us; error 43 us; skew 500 ppm
Aug 11 02:54:44 dist-test-slave-9gcw krb5kdc[28987](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416884, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.138.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 02:54:44.198448 29261 init.cc:377] Logged in from keytab as kudu/127.12.138.194@KRBTEST.COM (short username kudu)
I20260811 02:54:44.199350 29261 webserver.cc:460] Webserver started at http://127.12.138.194:38055/ using document root <none> and password file <none>
I20260811 02:54:44.199900 29261 fs_manager.cc:359] Metadata directory not provided
I20260811 02:54:44.199960 29261 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 02:54:44.200140 29261 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 02:54:44.202283 29261 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786416707705494-12843-0/minicluster-data/ts-1/data/instance:
uuid: "8a43edab73754e09a4b5e387d083f139"
format_stamp: "Formatted at 2026-08-11 02:54:44 on dist-test-slave-9gcw"
server_key: "0b24223102881f0eeb80a43fe932ab86"
server_key_iv: "1d7ad0a13c8dd59c653d43d0d1ab63ac"
server_key_version: "encryptionkey@0"
I20260811 02:54:44.202796 29261 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786416707705494-12843-0/minicluster-data/ts-1/wal/instance:
uuid: "8a43edab73754e09a4b5e387d083f139"
format_stamp: "Formatted at 2026-08-11 02:54:44 on dist-test-slave-9gcw"
server_key: "0b24223102881f0eeb80a43fe932ab86"
server_key_iv: "1d7ad0a13c8dd59c653d43d0d1ab63ac"
server_key_version: "encryptionkey@0"
I20260811 02:54:44.206591 29261 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260811 02:54:44.209028 29278 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 02:54:44.210394 29261 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260811 02:54:44.210520 29261 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786416707705494-12843-0/minicluster-data/ts-1/data,/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786416707705494-12843-0/minicluster-data/ts-1/wal
uuid: "8a43edab73754e09a4b5e387d083f139"
format_stamp: "Formatted at 2026-08-11 02:54:44 on dist-test-slave-9gcw"
server_key: "0b24223102881f0eeb80a43fe932ab86"
server_key_iv: "1d7ad0a13c8dd59c653d43d0d1ab63ac"
server_key_version: "encryptionkey@0"
I20260811 02:54:44.210624 29261 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786416707705494-12843-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786416707705494-12843-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786416707705494-12843-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 02:54:44.234417 29261 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 02:54:44.238150 29261 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 02:54:44.238350 29261 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 02:54:44.238957 29261 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 02:54:44.239919 29261 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 02:54:44.239977 29261 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 02:54:44.240021 29261 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 02:54:44.240047 29261 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 02:54:44.250237 29261 rpc_server.cc:307] RPC server started. Bound to: 127.12.138.194:45119
I20260811 02:54:44.250257 29410 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.138.194:45119 every 8 connection(s)
I20260811 02:54:44.251262 29261 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786416707705494-12843-0/minicluster-data/ts-1/data/info.pb
I20260811 02:54:44.254488 12843 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDawEwS/build/debug/bin/kudu as pid 29261
I20260811 02:54:44.254609 12843 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786416707705494-12843-0/minicluster-data/ts-1/wal/instance
I20260811 02:54:44.254967 12843 external_mini_cluster.cc:1428] Setting key 210e081b28a23524c1aa8e15c31881ac
Aug 11 02:54:44 dist-test-slave-9gcw krb5kdc[28987](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416884, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.138.194@KRBTEST.COM for kudu/127.12.138.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.12.138.195@KRBTEST.COM; defaulting to no policy
I20260811 02:54:44.267313 29082 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:54:44.253229 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.138.194:53149 (local address 127.12.138.254:45409)
0811 02:54:44.253384 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0811 02:54:44.253389 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 02:54:44.254347 (+ 958us) server_negotiation.cc:407] Connection header received
0811 02:54:44.256252 (+ 1905us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:54:44.256256 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:54:44.256315 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:54:44.256402 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:54:44.257925 (+ 1523us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:54:44.258875 (+ 950us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:54:44.260657 (+ 1782us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:54:44.260904 (+ 247us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:54:44.261021 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:54:44.263642 (+ 2621us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:54:44.263645 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:54:44.263647 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:54:44.263671 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:54:44.265095 (+ 1424us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:54:44.265683 (+ 588us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:54:44.265687 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:54:44.265689 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:54:44.265772 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:54:44.266122 (+ 350us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:54:44.266125 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:54:44.266127 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:54:44.266345 (+ 218us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:54:44.266447 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0811 02:54:44.267029 (+ 582us) server_negotiation.cc:299] Negotiation successful
0811 02:54:44.267144 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260811 02:54:44.268252 29414 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:54:44.253604 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.138.254:45409 (local address 127.12.138.194:53149)
0811 02:54:44.254157 (+ 553us) negotiation.cc:107] Waiting for socket to connect
0811 02:54:44.254205 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0811 02:54:44.256016 (+ 1811us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:54:44.256602 (+ 586us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:54:44.256609 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:54:44.257218 (+ 609us) client_negotiation.cc:189] Negotiated authn=SASL
0811 02:54:44.257752 (+ 534us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:54:44.257769 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:54:44.259043 (+ 1274us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:54:44.259049 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:54:44.260529 (+ 1480us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:54:44.260540 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:54:44.261062 (+ 522us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:54:44.261067 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:54:44.261299 (+ 232us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:54:44.261972 (+ 673us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 02:54:44.261989 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 02:54:44.263464 (+ 1475us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 02:54:44.265275 (+ 1811us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:54:44.265281 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:54:44.265299 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:54:44.265572 (+ 273us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:54:44.265863 (+ 291us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:54:44.265867 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:54:44.265869 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:54:44.266016 (+ 147us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:54:44.266474 (+ 458us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 02:54:44.266483 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 02:54:44.266797 (+ 314us) client_negotiation.cc:769] Sending connection context
0811 02:54:44.266971 (+ 174us) client_negotiation.cc:240] Negotiation successful
0811 02:54:44.267329 (+ 358us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":300,"thread_start_us":131,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.138.195@KRBTEST.COM" created.
I20260811 02:54:44.269696 29411 heartbeater.cc:347] Connected to a master server at 127.12.138.254:45409
I20260811 02:54:44.269969 29411 heartbeater.cc:464] Registering TS with master...
I20260811 02:54:44.270570 29411 heartbeater.cc:511] Master 127.12.138.254:45409 requested a full tablet report, sending...
I20260811 02:54:44.271714 29039 ts_manager.cc:194] Registered new tserver with Master: 8a43edab73754e09a4b5e387d083f139 (127.12.138.194:45119)
I20260811 02:54:44.272307 29039 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.12.138.194@KRBTEST.COM'} at 127.12.138.194:53149
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.138.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786416707705494-12843-0/krb5kdc/kudu_127.12.138.195.keytab.
Entry for principal kudu/127.12.138.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786416707705494-12843-0/krb5kdc/kudu_127.12.138.195.keytab.
I20260811 02:54:44.289023 12843 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDawEwS/build/debug/bin/kudu
/tmp/dist-test-taskDawEwS/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786416707705494-12843-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786416707705494-12843-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786416707705494-12843-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786416707705494-12843-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.138.195:0
--local_ip_for_outbound_sockets=127.12.138.195
--webserver_interface=127.12.138.195
--webserver_port=0
--tserver_master_addrs=127.12.138.254:45409
--keytab_file=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786416707705494-12843-0/krb5kdc/kudu_127.12.138.195.keytab
--principal=kudu/127.12.138.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.138.212:45835
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786416707705494-12843-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786416707705494-12843-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786416707705494-12843-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 02:54:44.408900 29418 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 02:54:44.409242 29418 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 02:54:44.409291 29418 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 02:54:44.413327 29418 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 02:54:44.413385 29418 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 02:54:44.413443 29418 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.138.195
I20260811 02:54:44.418848 29418 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.138.212:45835
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786416707705494-12843-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786416707705494-12843-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786416707705494-12843-0/krb5kdc/kudu_127.12.138.195.keytab
--principal=kudu/127.12.138.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.138.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786416707705494-12843-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.12.138.195
--webserver_port=0
--tserver_master_addrs=127.12.138.254:45409
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29418
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.138.195
--log_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786416707705494-12843-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 02:09:49 UTC on ccycloud.quasar-fsnqqc.root.comops.site
build id 4579570
I20260811 02:54:44.419862 29418 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 02:54:44.420837 29418 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 02:54:44.430933 29423 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:54:44.430985 29424 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:54:44.430985 29426 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:54:44.431186 29418 server_base.cc:1034] running on GCE node
I20260811 02:54:44.431594 29418 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 02:54:44.432221 29418 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 02:54:44.433373 29418 hybrid_clock.cc:648] HybridClock initialized: now 1786416884433339 us; error 46 us; skew 500 ppm
Aug 11 02:54:44 dist-test-slave-9gcw krb5kdc[28987](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416884, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.138.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 02:54:44.436141 29418 init.cc:377] Logged in from keytab as kudu/127.12.138.195@KRBTEST.COM (short username kudu)
I20260811 02:54:44.437036 29418 webserver.cc:460] Webserver started at http://127.12.138.195:42981/ using document root <none> and password file <none>
I20260811 02:54:44.437667 29418 fs_manager.cc:359] Metadata directory not provided
I20260811 02:54:44.437738 29418 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 02:54:44.437974 29418 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 02:54:44.440085 29418 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786416707705494-12843-0/minicluster-data/ts-2/data/instance:
uuid: "11f70dbb696346a491c8b80eaf7ce67f"
format_stamp: "Formatted at 2026-08-11 02:54:44 on dist-test-slave-9gcw"
server_key: "0eacae20ba2b9cf328c5a1f43f4d883a"
server_key_iv: "85600423a907dc8209ef1b09bcff9144"
server_key_version: "encryptionkey@0"
I20260811 02:54:44.440703 29418 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786416707705494-12843-0/minicluster-data/ts-2/wal/instance:
uuid: "11f70dbb696346a491c8b80eaf7ce67f"
format_stamp: "Formatted at 2026-08-11 02:54:44 on dist-test-slave-9gcw"
server_key: "0eacae20ba2b9cf328c5a1f43f4d883a"
server_key_iv: "85600423a907dc8209ef1b09bcff9144"
server_key_version: "encryptionkey@0"
I20260811 02:54:44.444664 29418 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260811 02:54:44.447445 29433 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 02:54:44.448873 29418 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260811 02:54:44.449035 29418 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786416707705494-12843-0/minicluster-data/ts-2/data,/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786416707705494-12843-0/minicluster-data/ts-2/wal
uuid: "11f70dbb696346a491c8b80eaf7ce67f"
format_stamp: "Formatted at 2026-08-11 02:54:44 on dist-test-slave-9gcw"
server_key: "0eacae20ba2b9cf328c5a1f43f4d883a"
server_key_iv: "85600423a907dc8209ef1b09bcff9144"
server_key_version: "encryptionkey@0"
I20260811 02:54:44.449209 29418 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786416707705494-12843-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786416707705494-12843-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786416707705494-12843-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 02:54:44.466631 29418 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 02:54:44.470551 29418 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 02:54:44.470884 29418 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 02:54:44.471573 29418 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 02:54:44.472481 29418 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 02:54:44.472554 29418 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 02:54:44.472630 29418 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 02:54:44.472747 29418 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 02:54:44.482060 29418 rpc_server.cc:307] RPC server started. Bound to: 127.12.138.195:46749
I20260811 02:54:44.482087 29563 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.138.195:46749 every 8 connection(s)
I20260811 02:54:44.483021 29418 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786416707705494-12843-0/minicluster-data/ts-2/data/info.pb
I20260811 02:54:44.487084 12843 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDawEwS/build/debug/bin/kudu as pid 29418
I20260811 02:54:44.487206 12843 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786416707705494-12843-0/minicluster-data/ts-2/wal/instance
I20260811 02:54:44.487496 12843 external_mini_cluster.cc:1428] Setting key 2486840a9001b6d902ef8bde1567a210
Aug 11 02:54:44 dist-test-slave-9gcw krb5kdc[28987](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416884, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.138.195@KRBTEST.COM for kudu/127.12.138.254@KRBTEST.COM
I20260811 02:54:44.497028 29082 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:54:44.484884 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.138.195:38319 (local address 127.12.138.254:45409)
0811 02:54:44.485024 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0811 02:54:44.485028 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 02:54:44.485954 (+ 926us) server_negotiation.cc:407] Connection header received
0811 02:54:44.487416 (+ 1462us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:54:44.487420 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:54:44.487489 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:54:44.487588 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:54:44.489099 (+ 1511us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:54:44.489806 (+ 707us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:54:44.490950 (+ 1144us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:54:44.491173 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:54:44.491263 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:54:44.493788 (+ 2525us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:54:44.493792 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:54:44.493794 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:54:44.493818 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:54:44.495284 (+ 1466us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:54:44.495758 (+ 474us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:54:44.495760 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:54:44.495762 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:54:44.495836 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:54:44.496117 (+ 281us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:54:44.496119 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:54:44.496120 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:54:44.496294 (+ 174us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:54:44.496368 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0811 02:54:44.496802 (+ 434us) server_negotiation.cc:299] Negotiation successful
0811 02:54:44.496903 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260811 02:54:44.497960 29566 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:54:44.485329 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.138.254:45409 (local address 127.12.138.195:38319)
0811 02:54:44.485782 (+ 453us) negotiation.cc:107] Waiting for socket to connect
0811 02:54:44.485838 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0811 02:54:44.487167 (+ 1329us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:54:44.487765 (+ 598us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:54:44.487773 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:54:44.488277 (+ 504us) client_negotiation.cc:189] Negotiated authn=SASL
0811 02:54:44.488945 (+ 668us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:54:44.488968 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:54:44.489941 (+ 973us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:54:44.489943 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:54:44.490841 (+ 898us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:54:44.490850 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:54:44.491305 (+ 455us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:54:44.491307 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:54:44.491462 (+ 155us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:54:44.492014 (+ 552us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 02:54:44.492031 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 02:54:44.493595 (+ 1564us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 02:54:44.495384 (+ 1789us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:54:44.495389 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:54:44.495404 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:54:44.495645 (+ 241us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:54:44.495913 (+ 268us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:54:44.495915 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:54:44.495917 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:54:44.496044 (+ 127us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:54:44.496376 (+ 332us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 02:54:44.496380 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 02:54:44.496605 (+ 225us) client_negotiation.cc:769] Sending connection context
0811 02:54:44.496763 (+ 158us) client_negotiation.cc:240] Negotiation successful
0811 02:54:44.497028 (+ 265us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":252,"thread_start_us":107,"threads_started":1}
I20260811 02:54:44.499065 29564 heartbeater.cc:347] Connected to a master server at 127.12.138.254:45409
I20260811 02:54:44.499311 29564 heartbeater.cc:464] Registering TS with master...
I20260811 02:54:44.499800 29564 heartbeater.cc:511] Master 127.12.138.254:45409 requested a full tablet report, sending...
I20260811 02:54:44.500939 29039 ts_manager.cc:194] Registered new tserver with Master: 11f70dbb696346a491c8b80eaf7ce67f (127.12.138.195:46749)
I20260811 02:54:44.501573 12843 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 02:54:44.501729 29039 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.12.138.195@KRBTEST.COM'} at 127.12.138.195:38319
I20260811 02:54:44.510860 29082 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:54:44.503763 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58632 (local address 127.12.138.254:45409)
0811 02:54:44.503878 (+ 115us) server_negotiation.cc:206] Beginning negotiation
0811 02:54:44.503882 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 02:54:44.504041 (+ 159us) server_negotiation.cc:407] Connection header received
0811 02:54:44.504125 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:54:44.504127 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:54:44.504162 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:54:44.504225 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:54:44.504954 (+ 729us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:54:44.505528 (+ 574us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:54:44.506351 (+ 823us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:54:44.506550 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:54:44.506624 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:54:44.507914 (+ 1290us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:54:44.507916 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:54:44.507918 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:54:44.507938 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:54:44.509414 (+ 1476us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:54:44.509789 (+ 375us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:54:44.509792 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:54:44.509793 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:54:44.509867 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:54:44.510085 (+ 218us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:54:44.510087 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:54:44.510088 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:54:44.510285 (+ 197us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:54:44.510400 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0811 02:54:44.510644 (+ 244us) server_negotiation.cc:299] Negotiation successful
0811 02:54:44.510742 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260811 02:54:44.513742 29039 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58632:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 02:54:44.516911 29039 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 02:54:44.533288 29580 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:54:44.526849 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44726 (local address 127.12.138.195:46749)
0811 02:54:44.527490 (+ 641us) server_negotiation.cc:206] Beginning negotiation
0811 02:54:44.527503 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0811 02:54:44.527525 (+ 22us) server_negotiation.cc:407] Connection header received
0811 02:54:44.527606 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:54:44.527612 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:54:44.527814 (+ 202us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:54:44.528405 (+ 591us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 02:54:44.528569 (+ 164us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:54:44.529794 (+ 1225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:54:44.531966 (+ 2172us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:54:44.532518 (+ 552us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:54:44.532655 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:54:44.532801 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0811 02:54:44.533004 (+ 203us) server_negotiation.cc:299] Negotiation successful
0811 02:54:44.533103 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":535,"thread_start_us":165,"threads_started":1}
I20260811 02:54:44.533321 29578 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:54:44.526829 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.138.193:40085 (local address 127.0.0.1:45546)
0811 02:54:44.527760 (+ 931us) negotiation.cc:107] Waiting for socket to connect
0811 02:54:44.527775 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 02:54:44.527867 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:54:44.528394 (+ 527us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:54:44.528397 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:54:44.528414 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 02:54:44.528534 (+ 120us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:54:44.528545 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:54:44.529694 (+ 1149us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:54:44.529699 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:54:44.531513 (+ 1814us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:54:44.531524 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:54:44.532899 (+ 1375us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:54:44.532906 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:54:44.533014 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:54:44.533041 (+ 27us) client_negotiation.cc:769] Sending connection context
0811 02:54:44.533114 (+ 73us) client_negotiation.cc:240] Negotiation successful
0811 02:54:44.533208 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":871,"mutex_wait_us":173,"thread_start_us":94,"threads_started":1}
I20260811 02:54:44.533279 29579 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:54:44.526674 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.138.195:46749 (local address 127.0.0.1:44726)
0811 02:54:44.527081 (+ 407us) negotiation.cc:107] Waiting for socket to connect
0811 02:54:44.527097 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 02:54:44.527236 (+ 139us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:54:44.527978 (+ 742us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:54:44.527981 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:54:44.528000 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 02:54:44.528119 (+ 119us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:54:44.528175 (+ 56us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:54:44.529943 (+ 1768us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:54:44.529946 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:54:44.531861 (+ 1915us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:54:44.531871 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:54:44.532641 (+ 770us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:54:44.532644 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:54:44.532780 (+ 136us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:54:44.532842 (+ 62us) client_negotiation.cc:769] Sending connection context
0811 02:54:44.532969 (+ 127us) client_negotiation.cc:240] Negotiation successful
0811 02:54:44.533103 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":339,"mutex_wait_us":528,"thread_start_us":113,"threads_started":1}
I20260811 02:54:44.533792 29582 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:54:44.527260 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45546 (local address 127.12.138.193:40085)
0811 02:54:44.527528 (+ 268us) server_negotiation.cc:206] Beginning negotiation
0811 02:54:44.527535 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 02:54:44.527974 (+ 439us) server_negotiation.cc:407] Connection header received
0811 02:54:44.528065 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:54:44.528084 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:54:44.528262 (+ 178us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:54:44.528419 (+ 157us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 02:54:44.528658 (+ 239us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:54:44.529545 (+ 887us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:54:44.531687 (+ 2142us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:54:44.532536 (+ 849us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:54:44.532673 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:54:44.532829 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0811 02:54:44.533519 (+ 690us) server_negotiation.cc:299] Negotiation successful
0811 02:54:44.533646 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":172,"thread_start_us":106,"threads_started":1}
I20260811 02:54:44.533831 29577 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:54:44.526234 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39894 (local address 127.12.138.194:45119)
0811 02:54:44.527136 (+ 902us) server_negotiation.cc:206] Beginning negotiation
0811 02:54:44.527153 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0811 02:54:44.527184 (+ 31us) server_negotiation.cc:407] Connection header received
0811 02:54:44.527276 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:54:44.527285 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:54:44.527525 (+ 240us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:54:44.528181 (+ 656us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 02:54:44.528375 (+ 194us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:54:44.529743 (+ 1368us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:54:44.531496 (+ 1753us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:54:44.532435 (+ 939us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:54:44.533413 (+ 978us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:54:44.533556 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0811 02:54:44.533603 (+ 47us) server_negotiation.cc:299] Negotiation successful
0811 02:54:44.533688 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":763,"thread_start_us":106,"threads_started":1}
I20260811 02:54:44.533311 29576 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:54:44.526101 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.138.194:45119 (local address 127.0.0.1:39894)
0811 02:54:44.526475 (+ 374us) negotiation.cc:107] Waiting for socket to connect
0811 02:54:44.526536 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0811 02:54:44.526674 (+ 138us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:54:44.527735 (+ 1061us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:54:44.527742 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:54:44.527767 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 02:54:44.527975 (+ 208us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:54:44.527984 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:54:44.529932 (+ 1948us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:54:44.529936 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:54:44.531355 (+ 1419us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:54:44.531365 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:54:44.532607 (+ 1242us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:54:44.532611 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:54:44.532796 (+ 185us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:54:44.532841 (+ 45us) client_negotiation.cc:769] Sending connection context
0811 02:54:44.532936 (+ 95us) client_negotiation.cc:240] Negotiation successful
0811 02:54:44.533116 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":174,"mutex_wait_us":77,"thread_start_us":79,"threads_started":1}
I20260811 02:54:44.536755 29180 tablet_service.cc:1467] Processing CreateTablet for tablet 42df1435ae4745df9f91aab1e7568270 (DEFAULT_TABLE table=test-table [id=a8015fa6fe2842dc8ecc81a639578020]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 02:54:44.537137 29330 tablet_service.cc:1467] Processing CreateTablet for tablet 42df1435ae4745df9f91aab1e7568270 (DEFAULT_TABLE table=test-table [id=a8015fa6fe2842dc8ecc81a639578020]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 02:54:44.537108 29486 tablet_service.cc:1467] Processing CreateTablet for tablet 42df1435ae4745df9f91aab1e7568270 (DEFAULT_TABLE table=test-table [id=a8015fa6fe2842dc8ecc81a639578020]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 02:54:44.537901 29180 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 42df1435ae4745df9f91aab1e7568270. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 02:54:44.538265 29330 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 42df1435ae4745df9f91aab1e7568270. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 02:54:44.538273 29486 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 42df1435ae4745df9f91aab1e7568270. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 02:54:44.543833 29583 tablet_bootstrap.cc:492] T 42df1435ae4745df9f91aab1e7568270 P 13a904a6ad6a46a78dd222ee22eaea3c: Bootstrap starting.
I20260811 02:54:44.545948 29584 tablet_bootstrap.cc:492] T 42df1435ae4745df9f91aab1e7568270 P 11f70dbb696346a491c8b80eaf7ce67f: Bootstrap starting.
I20260811 02:54:44.546653 29583 tablet_bootstrap.cc:654] T 42df1435ae4745df9f91aab1e7568270 P 13a904a6ad6a46a78dd222ee22eaea3c: Neither blocks nor log segments found. Creating new log.
I20260811 02:54:44.546846 29586 tablet_bootstrap.cc:492] T 42df1435ae4745df9f91aab1e7568270 P 8a43edab73754e09a4b5e387d083f139: Bootstrap starting.
I20260811 02:54:44.547569 29583 log.cc:824] T 42df1435ae4745df9f91aab1e7568270 P 13a904a6ad6a46a78dd222ee22eaea3c: Log is configured to *not* fsync() on all Append() calls
I20260811 02:54:44.548242 29584 tablet_bootstrap.cc:654] T 42df1435ae4745df9f91aab1e7568270 P 11f70dbb696346a491c8b80eaf7ce67f: Neither blocks nor log segments found. Creating new log.
I20260811 02:54:44.549526 29584 log.cc:824] T 42df1435ae4745df9f91aab1e7568270 P 11f70dbb696346a491c8b80eaf7ce67f: Log is configured to *not* fsync() on all Append() calls
I20260811 02:54:44.549707 29586 tablet_bootstrap.cc:654] T 42df1435ae4745df9f91aab1e7568270 P 8a43edab73754e09a4b5e387d083f139: Neither blocks nor log segments found. Creating new log.
I20260811 02:54:44.550612 29586 log.cc:824] T 42df1435ae4745df9f91aab1e7568270 P 8a43edab73754e09a4b5e387d083f139: Log is configured to *not* fsync() on all Append() calls
I20260811 02:54:44.550750 29583 tablet_bootstrap.cc:492] T 42df1435ae4745df9f91aab1e7568270 P 13a904a6ad6a46a78dd222ee22eaea3c: No bootstrap required, opened a new log
I20260811 02:54:44.550971 29583 ts_tablet_manager.cc:1397] T 42df1435ae4745df9f91aab1e7568270 P 13a904a6ad6a46a78dd222ee22eaea3c: Time spent bootstrapping tablet: real 0.007s user 0.004s sys 0.000s
I20260811 02:54:44.551430 29584 tablet_bootstrap.cc:492] T 42df1435ae4745df9f91aab1e7568270 P 11f70dbb696346a491c8b80eaf7ce67f: No bootstrap required, opened a new log
I20260811 02:54:44.551620 29584 ts_tablet_manager.cc:1397] T 42df1435ae4745df9f91aab1e7568270 P 11f70dbb696346a491c8b80eaf7ce67f: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260811 02:54:44.552800 29586 tablet_bootstrap.cc:492] T 42df1435ae4745df9f91aab1e7568270 P 8a43edab73754e09a4b5e387d083f139: No bootstrap required, opened a new log
I20260811 02:54:44.553020 29586 ts_tablet_manager.cc:1397] T 42df1435ae4745df9f91aab1e7568270 P 8a43edab73754e09a4b5e387d083f139: Time spent bootstrapping tablet: real 0.006s user 0.000s sys 0.004s
I20260811 02:54:44.554899 29584 raft_consensus.cc:348] T 42df1435ae4745df9f91aab1e7568270 P 11f70dbb696346a491c8b80eaf7ce67f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a43edab73754e09a4b5e387d083f139" member_type: VOTER last_known_addr { host: "127.12.138.194" port: 45119 } } peers { permanent_uuid: "11f70dbb696346a491c8b80eaf7ce67f" member_type: VOTER last_known_addr { host: "127.12.138.195" port: 46749 } } peers { permanent_uuid: "13a904a6ad6a46a78dd222ee22eaea3c" member_type: VOTER last_known_addr { host: "127.12.138.193" port: 40085 } }
I20260811 02:54:44.555135 29584 raft_consensus.cc:374] T 42df1435ae4745df9f91aab1e7568270 P 11f70dbb696346a491c8b80eaf7ce67f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 02:54:44.555187 29584 raft_consensus.cc:729] T 42df1435ae4745df9f91aab1e7568270 P 11f70dbb696346a491c8b80eaf7ce67f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 11f70dbb696346a491c8b80eaf7ce67f, State: Initialized, Role: FOLLOWER
I20260811 02:54:44.555639 29584 consensus_queue.cc:260] T 42df1435ae4745df9f91aab1e7568270 P 11f70dbb696346a491c8b80eaf7ce67f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a43edab73754e09a4b5e387d083f139" member_type: VOTER last_known_addr { host: "127.12.138.194" port: 45119 } } peers { permanent_uuid: "11f70dbb696346a491c8b80eaf7ce67f" member_type: VOTER last_known_addr { host: "127.12.138.195" port: 46749 } } peers { permanent_uuid: "13a904a6ad6a46a78dd222ee22eaea3c" member_type: VOTER last_known_addr { host: "127.12.138.193" port: 40085 } }
I20260811 02:54:44.556320 29564 heartbeater.cc:503] Master 127.12.138.254:45409 was elected leader, sending a full tablet report...
I20260811 02:54:44.556187 29583 raft_consensus.cc:348] T 42df1435ae4745df9f91aab1e7568270 P 13a904a6ad6a46a78dd222ee22eaea3c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a43edab73754e09a4b5e387d083f139" member_type: VOTER last_known_addr { host: "127.12.138.194" port: 45119 } } peers { permanent_uuid: "11f70dbb696346a491c8b80eaf7ce67f" member_type: VOTER last_known_addr { host: "127.12.138.195" port: 46749 } } peers { permanent_uuid: "13a904a6ad6a46a78dd222ee22eaea3c" member_type: VOTER last_known_addr { host: "127.12.138.193" port: 40085 } }
I20260811 02:54:44.556440 29583 raft_consensus.cc:374] T 42df1435ae4745df9f91aab1e7568270 P 13a904a6ad6a46a78dd222ee22eaea3c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 02:54:44.556509 29583 raft_consensus.cc:729] T 42df1435ae4745df9f91aab1e7568270 P 13a904a6ad6a46a78dd222ee22eaea3c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 13a904a6ad6a46a78dd222ee22eaea3c, State: Initialized, Role: FOLLOWER
I20260811 02:54:44.556672 29584 ts_tablet_manager.cc:1428] T 42df1435ae4745df9f91aab1e7568270 P 11f70dbb696346a491c8b80eaf7ce67f: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260811 02:54:44.556949 29583 consensus_queue.cc:260] T 42df1435ae4745df9f91aab1e7568270 P 13a904a6ad6a46a78dd222ee22eaea3c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a43edab73754e09a4b5e387d083f139" member_type: VOTER last_known_addr { host: "127.12.138.194" port: 45119 } } peers { permanent_uuid: "11f70dbb696346a491c8b80eaf7ce67f" member_type: VOTER last_known_addr { host: "127.12.138.195" port: 46749 } } peers { permanent_uuid: "13a904a6ad6a46a78dd222ee22eaea3c" member_type: VOTER last_known_addr { host: "127.12.138.193" port: 40085 } }
I20260811 02:54:44.557276 29586 raft_consensus.cc:348] T 42df1435ae4745df9f91aab1e7568270 P 8a43edab73754e09a4b5e387d083f139 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a43edab73754e09a4b5e387d083f139" member_type: VOTER last_known_addr { host: "127.12.138.194" port: 45119 } } peers { permanent_uuid: "11f70dbb696346a491c8b80eaf7ce67f" member_type: VOTER last_known_addr { host: "127.12.138.195" port: 46749 } } peers { permanent_uuid: "13a904a6ad6a46a78dd222ee22eaea3c" member_type: VOTER last_known_addr { host: "127.12.138.193" port: 40085 } }
I20260811 02:54:44.557546 29586 raft_consensus.cc:374] T 42df1435ae4745df9f91aab1e7568270 P 8a43edab73754e09a4b5e387d083f139 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 02:54:44.557617 29586 raft_consensus.cc:729] T 42df1435ae4745df9f91aab1e7568270 P 8a43edab73754e09a4b5e387d083f139 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8a43edab73754e09a4b5e387d083f139, State: Initialized, Role: FOLLOWER
I20260811 02:54:44.557693 29255 heartbeater.cc:503] Master 127.12.138.254:45409 was elected leader, sending a full tablet report...
I20260811 02:54:44.558032 29583 ts_tablet_manager.cc:1428] T 42df1435ae4745df9f91aab1e7568270 P 13a904a6ad6a46a78dd222ee22eaea3c: Time spent starting tablet: real 0.007s user 0.002s sys 0.006s
I20260811 02:54:44.558041 29586 consensus_queue.cc:260] T 42df1435ae4745df9f91aab1e7568270 P 8a43edab73754e09a4b5e387d083f139 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a43edab73754e09a4b5e387d083f139" member_type: VOTER last_known_addr { host: "127.12.138.194" port: 45119 } } peers { permanent_uuid: "11f70dbb696346a491c8b80eaf7ce67f" member_type: VOTER last_known_addr { host: "127.12.138.195" port: 46749 } } peers { permanent_uuid: "13a904a6ad6a46a78dd222ee22eaea3c" member_type: VOTER last_known_addr { host: "127.12.138.193" port: 40085 } }
I20260811 02:54:44.558722 29411 heartbeater.cc:503] Master 127.12.138.254:45409 was elected leader, sending a full tablet report...
I20260811 02:54:44.558992 29586 ts_tablet_manager.cc:1428] T 42df1435ae4745df9f91aab1e7568270 P 8a43edab73754e09a4b5e387d083f139: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260811 02:54:44.622761 29595 raft_consensus.cc:482] T 42df1435ae4745df9f91aab1e7568270 P 8a43edab73754e09a4b5e387d083f139 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 02:54:44.623052 29595 raft_consensus.cc:504] T 42df1435ae4745df9f91aab1e7568270 P 8a43edab73754e09a4b5e387d083f139 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a43edab73754e09a4b5e387d083f139" member_type: VOTER last_known_addr { host: "127.12.138.194" port: 45119 } } peers { permanent_uuid: "11f70dbb696346a491c8b80eaf7ce67f" member_type: VOTER last_known_addr { host: "127.12.138.195" port: 46749 } } peers { permanent_uuid: "13a904a6ad6a46a78dd222ee22eaea3c" member_type: VOTER last_known_addr { host: "127.12.138.193" port: 40085 } }
I20260811 02:54:44.624279 29595 leader_election.cc:288] T 42df1435ae4745df9f91aab1e7568270 P 8a43edab73754e09a4b5e387d083f139 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 11f70dbb696346a491c8b80eaf7ce67f (127.12.138.195:46749), 13a904a6ad6a46a78dd222ee22eaea3c (127.12.138.193:40085)
I20260811 02:54:44.628341 29580 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:54:44.624731 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.138.194:54623 (local address 127.12.138.195:46749)
0811 02:54:44.624884 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0811 02:54:44.624889 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 02:54:44.624906 (+ 17us) server_negotiation.cc:407] Connection header received
0811 02:54:44.625055 (+ 149us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:54:44.625058 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:54:44.625101 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:54:44.625218 (+ 117us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 02:54:44.625517 (+ 299us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:54:44.626068 (+ 551us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:54:44.627349 (+ 1281us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:54:44.627764 (+ 415us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:54:44.627903 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:54:44.627959 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0811 02:54:44.628132 (+ 173us) server_negotiation.cc:299] Negotiation successful
0811 02:54:44.628206 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260811 02:54:44.628364 29414 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:54:44.624576 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.138.195:46749 (local address 127.12.138.194:54623)
0811 02:54:44.624795 (+ 219us) negotiation.cc:107] Waiting for socket to connect
0811 02:54:44.624812 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 02:54:44.624914 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:54:44.625236 (+ 322us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:54:44.625239 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:54:44.625280 (+ 41us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 02:54:44.625403 (+ 123us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:54:44.625409 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:54:44.626188 (+ 779us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:54:44.626192 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:54:44.627207 (+ 1015us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:54:44.627218 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:54:44.627899 (+ 681us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:54:44.627903 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:54:44.628012 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:54:44.628030 (+ 18us) client_negotiation.cc:769] Sending connection context
0811 02:54:44.628114 (+ 84us) client_negotiation.cc:240] Negotiation successful
0811 02:54:44.628189 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":129,"spinlock_wait_cycles":1792}
I20260811 02:54:44.628397 29582 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:54:44.624821 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.138.194:43023 (local address 127.12.138.193:40085)
0811 02:54:44.624963 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0811 02:54:44.624967 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 02:54:44.625211 (+ 244us) server_negotiation.cc:407] Connection header received
0811 02:54:44.625422 (+ 211us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:54:44.625425 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:54:44.625468 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:54:44.625547 (+ 79us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 02:54:44.625831 (+ 284us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:54:44.626349 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:54:44.627393 (+ 1044us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:54:44.627817 (+ 424us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:54:44.627945 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:54:44.628001 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0811 02:54:44.628198 (+ 197us) server_negotiation.cc:299] Negotiation successful
0811 02:54:44.628275 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260811 02:54:44.628818 29596 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:54:44.624576 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.138.193:40085 (local address 127.12.138.194:43023)
0811 02:54:44.625068 (+ 492us) negotiation.cc:107] Waiting for socket to connect
0811 02:54:44.625080 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 02:54:44.625297 (+ 217us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:54:44.625570 (+ 273us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:54:44.625573 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:54:44.625589 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 02:54:44.625716 (+ 127us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:54:44.625722 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:54:44.626456 (+ 734us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:54:44.626462 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:54:44.627260 (+ 798us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:54:44.627269 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:54:44.627945 (+ 676us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:54:44.627949 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:54:44.628061 (+ 112us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:54:44.628089 (+ 28us) client_negotiation.cc:769] Sending connection context
0811 02:54:44.628179 (+ 90us) client_negotiation.cc:240] Negotiation successful
0811 02:54:44.628249 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":431,"mutex_wait_us":49,"spinlock_wait_cycles":32512,"thread_start_us":139,"threads_started":1}
I20260811 02:54:44.629122 29510 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "42df1435ae4745df9f91aab1e7568270" candidate_uuid: "8a43edab73754e09a4b5e387d083f139" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "11f70dbb696346a491c8b80eaf7ce67f" is_pre_election: true
I20260811 02:54:44.629465 29204 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "42df1435ae4745df9f91aab1e7568270" candidate_uuid: "8a43edab73754e09a4b5e387d083f139" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "13a904a6ad6a46a78dd222ee22eaea3c" is_pre_election: true
I20260811 02:54:44.629511 29510 raft_consensus.cc:2454] T 42df1435ae4745df9f91aab1e7568270 P 11f70dbb696346a491c8b80eaf7ce67f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a43edab73754e09a4b5e387d083f139 in term 0.
I20260811 02:54:44.629732 29204 raft_consensus.cc:2454] T 42df1435ae4745df9f91aab1e7568270 P 13a904a6ad6a46a78dd222ee22eaea3c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a43edab73754e09a4b5e387d083f139 in term 0.
I20260811 02:54:44.630038 29282 leader_election.cc:302] T 42df1435ae4745df9f91aab1e7568270 P 8a43edab73754e09a4b5e387d083f139 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11f70dbb696346a491c8b80eaf7ce67f, 8a43edab73754e09a4b5e387d083f139; no voters:
I20260811 02:54:44.630299 29595 raft_consensus.cc:2781] T 42df1435ae4745df9f91aab1e7568270 P 8a43edab73754e09a4b5e387d083f139 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 02:54:44.630388 29595 raft_consensus.cc:482] T 42df1435ae4745df9f91aab1e7568270 P 8a43edab73754e09a4b5e387d083f139 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 02:54:44.630432 29595 raft_consensus.cc:3037] T 42df1435ae4745df9f91aab1e7568270 P 8a43edab73754e09a4b5e387d083f139 [term 0 FOLLOWER]: Advancing to term 1
I20260811 02:54:44.631503 29595 raft_consensus.cc:504] T 42df1435ae4745df9f91aab1e7568270 P 8a43edab73754e09a4b5e387d083f139 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a43edab73754e09a4b5e387d083f139" member_type: VOTER last_known_addr { host: "127.12.138.194" port: 45119 } } peers { permanent_uuid: "11f70dbb696346a491c8b80eaf7ce67f" member_type: VOTER last_known_addr { host: "127.12.138.195" port: 46749 } } peers { permanent_uuid: "13a904a6ad6a46a78dd222ee22eaea3c" member_type: VOTER last_known_addr { host: "127.12.138.193" port: 40085 } }
I20260811 02:54:44.631870 29595 leader_election.cc:288] T 42df1435ae4745df9f91aab1e7568270 P 8a43edab73754e09a4b5e387d083f139 [CANDIDATE]: Term 1 election: Requested vote from peers 11f70dbb696346a491c8b80eaf7ce67f (127.12.138.195:46749), 13a904a6ad6a46a78dd222ee22eaea3c (127.12.138.193:40085)
I20260811 02:54:44.632207 29510 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "42df1435ae4745df9f91aab1e7568270" candidate_uuid: "8a43edab73754e09a4b5e387d083f139" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "11f70dbb696346a491c8b80eaf7ce67f"
I20260811 02:54:44.632257 29204 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "42df1435ae4745df9f91aab1e7568270" candidate_uuid: "8a43edab73754e09a4b5e387d083f139" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "13a904a6ad6a46a78dd222ee22eaea3c"
I20260811 02:54:44.632349 29510 raft_consensus.cc:3037] T 42df1435ae4745df9f91aab1e7568270 P 11f70dbb696346a491c8b80eaf7ce67f [term 0 FOLLOWER]: Advancing to term 1
I20260811 02:54:44.632373 29204 raft_consensus.cc:3037] T 42df1435ae4745df9f91aab1e7568270 P 13a904a6ad6a46a78dd222ee22eaea3c [term 0 FOLLOWER]: Advancing to term 1
I20260811 02:54:44.633760 29204 raft_consensus.cc:2454] T 42df1435ae4745df9f91aab1e7568270 P 13a904a6ad6a46a78dd222ee22eaea3c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a43edab73754e09a4b5e387d083f139 in term 1.
I20260811 02:54:44.633760 29510 raft_consensus.cc:2454] T 42df1435ae4745df9f91aab1e7568270 P 11f70dbb696346a491c8b80eaf7ce67f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a43edab73754e09a4b5e387d083f139 in term 1.
I20260811 02:54:44.634078 29281 leader_election.cc:302] T 42df1435ae4745df9f91aab1e7568270 P 8a43edab73754e09a4b5e387d083f139 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 13a904a6ad6a46a78dd222ee22eaea3c, 8a43edab73754e09a4b5e387d083f139; no voters:
I20260811 02:54:44.634282 29595 raft_consensus.cc:2781] T 42df1435ae4745df9f91aab1e7568270 P 8a43edab73754e09a4b5e387d083f139 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 02:54:44.634536 29595 raft_consensus.cc:686] T 42df1435ae4745df9f91aab1e7568270 P 8a43edab73754e09a4b5e387d083f139 [term 1 LEADER]: Becoming Leader. State: Replica: 8a43edab73754e09a4b5e387d083f139, State: Running, Role: LEADER
I20260811 02:54:44.634934 29595 consensus_queue.cc:237] T 42df1435ae4745df9f91aab1e7568270 P 8a43edab73754e09a4b5e387d083f139 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a43edab73754e09a4b5e387d083f139" member_type: VOTER last_known_addr { host: "127.12.138.194" port: 45119 } } peers { permanent_uuid: "11f70dbb696346a491c8b80eaf7ce67f" member_type: VOTER last_known_addr { host: "127.12.138.195" port: 46749 } } peers { permanent_uuid: "13a904a6ad6a46a78dd222ee22eaea3c" member_type: VOTER last_known_addr { host: "127.12.138.193" port: 40085 } }
I20260811 02:54:44.638448 29039 catalog_manager.cc:5495] T 42df1435ae4745df9f91aab1e7568270 P 8a43edab73754e09a4b5e387d083f139 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8a43edab73754e09a4b5e387d083f139 (127.12.138.194). New cstate: current_term: 1 leader_uuid: "8a43edab73754e09a4b5e387d083f139" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a43edab73754e09a4b5e387d083f139" member_type: VOTER last_known_addr { host: "127.12.138.194" port: 45119 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "11f70dbb696346a491c8b80eaf7ce67f" member_type: VOTER last_known_addr { host: "127.12.138.195" port: 46749 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "13a904a6ad6a46a78dd222ee22eaea3c" member_type: VOTER last_known_addr { host: "127.12.138.193" port: 40085 } health_report { overall_health: UNKNOWN } } }
I20260811 02:54:44.667522 29577 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:54:44.664095 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39898 (local address 127.12.138.194:45119)
0811 02:54:44.664264 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0811 02:54:44.664268 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 02:54:44.664281 (+ 13us) server_negotiation.cc:407] Connection header received
0811 02:54:44.664366 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:54:44.664368 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:54:44.664426 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:54:44.664531 (+ 105us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 02:54:44.664747 (+ 216us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:54:44.665312 (+ 565us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:54:44.666184 (+ 872us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:54:44.666402 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:54:44.666488 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:54:44.666733 (+ 245us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 02:54:44.667187 (+ 454us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 02:54:44.667281 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0811 02:54:44.667355 (+ 74us) server_negotiation.cc:299] Negotiation successful
0811 02:54:44.667404 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260811 02:54:44.674049 29204 raft_consensus.cc:1261] T 42df1435ae4745df9f91aab1e7568270 P 13a904a6ad6a46a78dd222ee22eaea3c [term 1 FOLLOWER]: Refusing update from remote peer 8a43edab73754e09a4b5e387d083f139: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 02:54:44.674049 29510 raft_consensus.cc:1261] T 42df1435ae4745df9f91aab1e7568270 P 11f70dbb696346a491c8b80eaf7ce67f [term 1 FOLLOWER]: Refusing update from remote peer 8a43edab73754e09a4b5e387d083f139: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 02:54:44.674880 29597 consensus_queue.cc:1035] T 42df1435ae4745df9f91aab1e7568270 P 8a43edab73754e09a4b5e387d083f139 [LEADER]: Connected to new peer: Peer: permanent_uuid: "13a904a6ad6a46a78dd222ee22eaea3c" member_type: VOTER last_known_addr { host: "127.12.138.193" port: 40085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 02:54:44.675037 29595 consensus_queue.cc:1035] T 42df1435ae4745df9f91aab1e7568270 P 8a43edab73754e09a4b5e387d083f139 [LEADER]: Connected to new peer: Peer: permanent_uuid: "11f70dbb696346a491c8b80eaf7ce67f" member_type: VOTER last_known_addr { host: "127.12.138.195" port: 46749 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 02:54:44.685253 29601 mvcc.cc:205] Tried to move back new op lower bound from 7317163559616704512 to 7317163559467880448. Current Snapshot: MvccSnapshot[applied={T|T < 7317163559616704512}]
I20260811 02:54:44.692916 29036 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58632:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 02:54:44.693107 29036 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58632:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 02:54:44.695647 29036 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 11144008f58842b883c076177aaad052: Sending DeleteTablet for 3 replicas of tablet 42df1435ae4745df9f91aab1e7568270
I20260811 02:54:44.696575 29330 tablet_service.cc:1514] Processing DeleteTablet for tablet 42df1435ae4745df9f91aab1e7568270 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 02:54:44 UTC) from {username='kudu', principal='kudu/127.12.138.254@KRBTEST.COM'} at 127.0.0.1:39894
I20260811 02:54:44.696586 29486 tablet_service.cc:1514] Processing DeleteTablet for tablet 42df1435ae4745df9f91aab1e7568270 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 02:54:44 UTC) from {username='kudu', principal='kudu/127.12.138.254@KRBTEST.COM'} at 127.0.0.1:44726
I20260811 02:54:44.696712 29180 tablet_service.cc:1514] Processing DeleteTablet for tablet 42df1435ae4745df9f91aab1e7568270 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 02:54:44 UTC) from {username='kudu', principal='kudu/127.12.138.254@KRBTEST.COM'} at 127.0.0.1:45546
I20260811 02:54:44.697268 29613 tablet_replica.cc:330] T 42df1435ae4745df9f91aab1e7568270 P 13a904a6ad6a46a78dd222ee22eaea3c: stopping tablet replica
I20260811 02:54:44.697387 12843 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDawEwS/build/debug/bin/kudu with pid 29109
I20260811 02:54:44.705344 29610 tablet_replica.cc:330] T 42df1435ae4745df9f91aab1e7568270 P 8a43edab73754e09a4b5e387d083f139: stopping tablet replica
I20260811 02:54:44.705890 29610 raft_consensus.cc:2229] T 42df1435ae4745df9f91aab1e7568270 P 8a43edab73754e09a4b5e387d083f139 [term 1 LEADER]: Raft consensus shutting down.
I20260811 02:54:44.706460 29610 raft_consensus.cc:2258] T 42df1435ae4745df9f91aab1e7568270 P 8a43edab73754e09a4b5e387d083f139 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 02:54:44.707255 29611 tablet_replica.cc:330] T 42df1435ae4745df9f91aab1e7568270 P 11f70dbb696346a491c8b80eaf7ce67f: stopping tablet replica
I20260811 02:54:44.707690 29611 raft_consensus.cc:2229] T 42df1435ae4745df9f91aab1e7568270 P 11f70dbb696346a491c8b80eaf7ce67f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 02:54:44.708148 29611 raft_consensus.cc:2258] T 42df1435ae4745df9f91aab1e7568270 P 11f70dbb696346a491c8b80eaf7ce67f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 02:54:44.710273 29611 ts_tablet_manager.cc:1905] T 42df1435ae4745df9f91aab1e7568270 P 11f70dbb696346a491c8b80eaf7ce67f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 02:54:44.714109 29611 ts_tablet_manager.cc:1918] T 42df1435ae4745df9f91aab1e7568270 P 11f70dbb696346a491c8b80eaf7ce67f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 02:54:44.714237 29611 log.cc:1196] T 42df1435ae4745df9f91aab1e7568270 P 11f70dbb696346a491c8b80eaf7ce67f: Deleting WAL directory at /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786416707705494-12843-0/minicluster-data/ts-2/wal/wals/42df1435ae4745df9f91aab1e7568270
I20260811 02:54:44.714654 29611 ts_tablet_manager.cc:1939] T 42df1435ae4745df9f91aab1e7568270 P 11f70dbb696346a491c8b80eaf7ce67f: Deleting consensus metadata
I20260811 02:54:44.716233 29024 catalog_manager.cc:4841] TS 11f70dbb696346a491c8b80eaf7ce67f (127.12.138.195:46749): tablet 42df1435ae4745df9f91aab1e7568270 (table test-table [id=a8015fa6fe2842dc8ecc81a639578020]) successfully deleted
W20260811 02:54:44.718920 29023 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.12.138.193:40085 (error 108)
I20260811 02:54:44.719226 12843 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDawEwS/build/debug/bin/kudu with pid 29261
I20260811 02:54:44.721698 29576 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:54:44.719863 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.138.193:40085 (local address 127.0.0.1:45550)
0811 02:54:44.721552 (+ 1689us) negotiation.cc:107] Waiting for socket to connect
0811 02:54:44.721619 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.138.193:40085: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1588}
W20260811 02:54:44.722184 29023 catalog_manager.cc:4584] TS 13a904a6ad6a46a78dd222ee22eaea3c (127.12.138.193:40085): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 42df1435ae4745df9f91aab1e7568270: Network error: Client connection negotiation failed: client connection to 127.12.138.193:40085: connect: Connection refused (error 111)
I20260811 02:54:44.729125 12843 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDawEwS/build/debug/bin/kudu with pid 29418
I20260811 02:54:44.729483 29576 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:54:44.729069 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.138.194:45119 (local address 127.0.0.1:39900)
0811 02:54:44.729314 (+ 245us) negotiation.cc:107] Waiting for socket to connect
0811 02:54:44.729380 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.138.194:45119: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
I20260811 02:54:44.737565 12843 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDawEwS/build/debug/bin/kudu with pid 29001
I20260811 02:54:44.747678 12843 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDawEwS/build/debug/bin/kudu
/tmp/dist-test-taskDawEwS/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786416707705494-12843-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786416707705494-12843-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786416707705494-12843-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786416707705494-12843-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.138.254:45409
--webserver_interface=127.12.138.254
--webserver_port=46571
--keytab_file=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786416707705494-12843-0/krb5kdc/kudu_127.12.138.254.keytab
--principal=kudu/127.12.138.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.138.212:45835
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.12.138.254:45409
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--ipki_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786416707705494-12843-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786416707705494-12843-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786416707705494-12843-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 02:54:44.862897 29615 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 02:54:44.863217 29615 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 02:54:44.863312 29615 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 02:54:44.867427 29615 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 02:54:44.867520 29615 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 02:54:44.867556 29615 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 02:54:44.867594 29615 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 02:54:44.867635 29615 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 02:54:44.873080 29615 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.138.212:45835
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786416707705494-12843-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786416707705494-12843-0/minicluster-data/master-0/wal
--ipki_private_key_password_cmd=echo foo
--ipki_ca_key_size=768
--master_addresses=127.12.138.254:45409
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786416707705494-12843-0/krb5kdc/kudu_127.12.138.254.keytab
--principal=kudu/127.12.138.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.138.254:45409
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786416707705494-12843-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.12.138.254
--webserver_port=46571
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29615
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786416707705494-12843-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 02:09:49 UTC on ccycloud.quasar-fsnqqc.root.comops.site
build id 4579570
I20260811 02:54:44.874339 29615 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 02:54:44.875458 29615 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 02:54:44.890054 29624 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:54:44.890091 29622 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:54:44.890048 29621 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:54:44.890506 29615 server_base.cc:1034] running on GCE node
I20260811 02:54:44.891152 29615 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 02:54:44.892544 29615 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 02:54:44.893728 29615 hybrid_clock.cc:648] HybridClock initialized: now 1786416884893710 us; error 31 us; skew 500 ppm
Aug 11 02:54:44 dist-test-slave-9gcw krb5kdc[28987](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416884, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.138.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 02:54:44.896698 29615 init.cc:377] Logged in from keytab as kudu/127.12.138.254@KRBTEST.COM (short username kudu)
I20260811 02:54:44.897778 29615 webserver.cc:460] Webserver started at http://127.12.138.254:46571/ using document root <none> and password file <none>
I20260811 02:54:44.898427 29615 fs_manager.cc:359] Metadata directory not provided
I20260811 02:54:44.898546 29615 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 02:54:44.902859 29615 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260811 02:54:44.905256 29631 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 02:54:44.906596 29615 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 02:54:44.906745 29615 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786416707705494-12843-0/minicluster-data/master-0/data,/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786416707705494-12843-0/minicluster-data/master-0/wal
uuid: "11144008f58842b883c076177aaad052"
format_stamp: "Formatted at 2026-08-11 02:54:43 on dist-test-slave-9gcw"
server_key: "2ee95749916bbaa0295a7c0d9ba9a695"
server_key_iv: "f7e08cdfd20e96649c8e5c4ac6e77e5d"
server_key_version: "encryptionkey@0"
I20260811 02:54:44.907428 29615 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786416707705494-12843-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786416707705494-12843-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786416707705494-12843-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 02:54:44.922433 29615 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 02:54:44.926951 29615 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 02:54:44.927178 29615 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 02:54:44.936126 29615 rpc_server.cc:307] RPC server started. Bound to: 127.12.138.254:45409
I20260811 02:54:44.936445 29685 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.138.254:45409 every 8 connection(s)
I20260811 02:54:44.937523 29615 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786416707705494-12843-0/minicluster-data/master-0/data/info.pb
I20260811 02:54:44.942224 29686 sys_catalog.cc:263] Verifying existing consensus state
I20260811 02:54:44.943863 29686 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 11144008f58842b883c076177aaad052: Bootstrap starting.
I20260811 02:54:44.945439 12843 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDawEwS/build/debug/bin/kudu as pid 29615
I20260811 02:54:44.945968 12843 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDawEwS/build/debug/bin/kudu
/tmp/dist-test-taskDawEwS/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786416707705494-12843-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786416707705494-12843-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786416707705494-12843-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786416707705494-12843-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.138.193:40085
--local_ip_for_outbound_sockets=127.12.138.193
--tserver_master_addrs=127.12.138.254:45409
--webserver_port=34989
--webserver_interface=127.12.138.193
--keytab_file=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786416707705494-12843-0/krb5kdc/kudu_127.12.138.193.keytab
--principal=kudu/127.12.138.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.138.212:45835
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786416707705494-12843-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786416707705494-12843-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786416707705494-12843-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 02:54:44.962419 29686 log.cc:824] T 00000000000000000000000000000000 P 11144008f58842b883c076177aaad052: Log is configured to *not* fsync() on all Append() calls
I20260811 02:54:44.972981 29686 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 11144008f58842b883c076177aaad052: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 02:54:44.973503 29686 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 11144008f58842b883c076177aaad052: Bootstrap complete.
I20260811 02:54:44.978425 29686 raft_consensus.cc:348] T 00000000000000000000000000000000 P 11144008f58842b883c076177aaad052 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "11144008f58842b883c076177aaad052" member_type: VOTER last_known_addr { host: "127.12.138.254" port: 45409 } }
I20260811 02:54:44.978907 29686 raft_consensus.cc:729] T 00000000000000000000000000000000 P 11144008f58842b883c076177aaad052 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 11144008f58842b883c076177aaad052, State: Initialized, Role: FOLLOWER
I20260811 02:54:44.979517 29686 consensus_queue.cc:260] T 00000000000000000000000000000000 P 11144008f58842b883c076177aaad052 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "11144008f58842b883c076177aaad052" member_type: VOTER last_known_addr { host: "127.12.138.254" port: 45409 } }
I20260811 02:54:44.979656 29686 raft_consensus.cc:388] T 00000000000000000000000000000000 P 11144008f58842b883c076177aaad052 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 02:54:44.979763 29686 raft_consensus.cc:482] T 00000000000000000000000000000000 P 11144008f58842b883c076177aaad052 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 02:54:44.979890 29686 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 11144008f58842b883c076177aaad052 [term 1 FOLLOWER]: Advancing to term 2
I20260811 02:54:44.982285 29686 raft_consensus.cc:504] T 00000000000000000000000000000000 P 11144008f58842b883c076177aaad052 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "11144008f58842b883c076177aaad052" member_type: VOTER last_known_addr { host: "127.12.138.254" port: 45409 } }
I20260811 02:54:44.982686 29686 leader_election.cc:302] T 00000000000000000000000000000000 P 11144008f58842b883c076177aaad052 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 11144008f58842b883c076177aaad052; no voters:
I20260811 02:54:44.983083 29686 leader_election.cc:288] T 00000000000000000000000000000000 P 11144008f58842b883c076177aaad052 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260811 02:54:44.983199 29691 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 11144008f58842b883c076177aaad052 [term 2 FOLLOWER]: Leader election won for term 2
I20260811 02:54:44.983415 29691 raft_consensus.cc:686] T 00000000000000000000000000000000 P 11144008f58842b883c076177aaad052 [term 2 LEADER]: Becoming Leader. State: Replica: 11144008f58842b883c076177aaad052, State: Running, Role: LEADER
I20260811 02:54:44.983750 29691 consensus_queue.cc:237] T 00000000000000000000000000000000 P 11144008f58842b883c076177aaad052 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "11144008f58842b883c076177aaad052" member_type: VOTER last_known_addr { host: "127.12.138.254" port: 45409 } }
I20260811 02:54:44.984782 29686 sys_catalog.cc:564] T 00000000000000000000000000000000 P 11144008f58842b883c076177aaad052 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 02:54:44.985584 29694 sys_catalog.cc:455] T 00000000000000000000000000000000 P 11144008f58842b883c076177aaad052 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 11144008f58842b883c076177aaad052. Latest consensus state: current_term: 2 leader_uuid: "11144008f58842b883c076177aaad052" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "11144008f58842b883c076177aaad052" member_type: VOTER last_known_addr { host: "127.12.138.254" port: 45409 } } }
I20260811 02:54:44.985597 29692 sys_catalog.cc:455] T 00000000000000000000000000000000 P 11144008f58842b883c076177aaad052 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "11144008f58842b883c076177aaad052" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "11144008f58842b883c076177aaad052" member_type: VOTER last_known_addr { host: "127.12.138.254" port: 45409 } } }
I20260811 02:54:44.985735 29692 sys_catalog.cc:458] T 00000000000000000000000000000000 P 11144008f58842b883c076177aaad052 [sys.catalog]: This master's current role is: LEADER
I20260811 02:54:44.985737 29694 sys_catalog.cc:458] T 00000000000000000000000000000000 P 11144008f58842b883c076177aaad052 [sys.catalog]: This master's current role is: LEADER
I20260811 02:54:44.986496 29699 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 02:54:44.991717 29699 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 02:54:44.992504 29699 catalog_manager.cc:1259] Loaded cluster ID: d19c1050b4cd4f2e8db89c9e58687fed
I20260811 02:54:44.992590 29699 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 02:54:44.993829 29699 catalog_manager.cc:1314] IPKI private key was stored in cleartext, attempting to encrypt it.
I20260811 02:54:44.996624 29699 catalog_manager.cc:1324] IPKI private key successfully encrypted.
I20260811 02:54:45.001303 29699 catalog_manager.cc:1504] Loading token signing keys...
I20260811 02:54:45.002094 29699 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 11144008f58842b883c076177aaad052: Loaded TSK: 0
I20260811 02:54:45.003206 29699 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260811 02:54:45.077636 29689 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 02:54:45.077916 29689 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 02:54:45.077960 29689 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 02:54:45.082069 29689 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 02:54:45.082127 29689 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 02:54:45.082192 29689 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.138.193
I20260811 02:54:45.087761 29689 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.138.212:45835
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786416707705494-12843-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786416707705494-12843-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786416707705494-12843-0/krb5kdc/kudu_127.12.138.193.keytab
--principal=kudu/127.12.138.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.138.193:40085
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786416707705494-12843-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.12.138.193
--webserver_port=34989
--tserver_master_addrs=127.12.138.254:45409
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29689
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.138.193
--log_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786416707705494-12843-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 02:09:49 UTC on ccycloud.quasar-fsnqqc.root.comops.site
build id 4579570
I20260811 02:54:45.088840 29689 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 02:54:45.089965 29689 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 02:54:45.100232 29719 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:54:45.100238 29722 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:54:45.100234 29720 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:54:45.100817 29689 server_base.cc:1034] running on GCE node
I20260811 02:54:45.101342 29689 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 02:54:45.102106 29689 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 02:54:45.103288 29689 hybrid_clock.cc:648] HybridClock initialized: now 1786416885103255 us; error 50 us; skew 500 ppm
Aug 11 02:54:45 dist-test-slave-9gcw krb5kdc[28987](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416885, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.138.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 02:54:45.106665 29689 init.cc:377] Logged in from keytab as kudu/127.12.138.193@KRBTEST.COM (short username kudu)
I20260811 02:54:45.107616 29689 webserver.cc:460] Webserver started at http://127.12.138.193:34989/ using document root <none> and password file <none>
I20260811 02:54:45.108225 29689 fs_manager.cc:359] Metadata directory not provided
I20260811 02:54:45.108340 29689 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 02:54:45.112748 29689 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.000s
I20260811 02:54:45.115365 29729 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 02:54:45.116751 29689 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260811 02:54:45.116891 29689 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786416707705494-12843-0/minicluster-data/ts-0/data,/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786416707705494-12843-0/minicluster-data/ts-0/wal
uuid: "13a904a6ad6a46a78dd222ee22eaea3c"
format_stamp: "Formatted at 2026-08-11 02:54:43 on dist-test-slave-9gcw"
server_key: "03fc6e3876c6257fb7637fbb9ccc8ff4"
server_key_iv: "68781f4cd4ccd5db115f71fa3f511505"
server_key_version: "encryptionkey@0"
I20260811 02:54:45.117573 29689 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786416707705494-12843-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786416707705494-12843-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786416707705494-12843-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 02:54:45.144345 29689 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 02:54:45.147939 29689 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 02:54:45.148175 29689 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 02:54:45.148895 29689 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 02:54:45.150137 29737 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260811 02:54:45.155201 29689 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260811 02:54:45.155290 29689 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.005s user 0.000s sys 0.000s
I20260811 02:54:45.155385 29689 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260811 02:54:45.157178 29689 ts_tablet_manager.cc:610] Registered 1 tablets
I20260811 02:54:45.157270 29689 ts_tablet_manager.cc:589] Time spent register tablets: real 0.002s user 0.002s sys 0.000s
I20260811 02:54:45.157578 29737 tablet_bootstrap.cc:492] T 42df1435ae4745df9f91aab1e7568270 P 13a904a6ad6a46a78dd222ee22eaea3c: Bootstrap starting.
I20260811 02:54:45.167999 29689 rpc_server.cc:307] RPC server started. Bound to: 127.12.138.193:40085
I20260811 02:54:45.168085 29859 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.138.193:40085 every 8 connection(s)
I20260811 02:54:45.169023 29689 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786416707705494-12843-0/minicluster-data/ts-0/data/info.pb
I20260811 02:54:45.174813 12843 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDawEwS/build/debug/bin/kudu as pid 29689
I20260811 02:54:45.175320 12843 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDawEwS/build/debug/bin/kudu
/tmp/dist-test-taskDawEwS/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786416707705494-12843-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786416707705494-12843-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786416707705494-12843-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786416707705494-12843-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.138.194:45119
--local_ip_for_outbound_sockets=127.12.138.194
--tserver_master_addrs=127.12.138.254:45409
--webserver_port=38055
--webserver_interface=127.12.138.194
--keytab_file=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786416707705494-12843-0/krb5kdc/kudu_127.12.138.194.keytab
--principal=kudu/127.12.138.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.138.212:45835
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786416707705494-12843-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786416707705494-12843-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786416707705494-12843-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 02:54:45.176802 29737 log.cc:824] T 42df1435ae4745df9f91aab1e7568270 P 13a904a6ad6a46a78dd222ee22eaea3c: Log is configured to *not* fsync() on all Append() calls
Aug 11 02:54:45 dist-test-slave-9gcw krb5kdc[28987](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416885, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.138.193@KRBTEST.COM for kudu/127.12.138.254@KRBTEST.COM
I20260811 02:54:45.185621 29737 tablet_bootstrap.cc:492] T 42df1435ae4745df9f91aab1e7568270 P 13a904a6ad6a46a78dd222ee22eaea3c: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 02:54:45.186088 29737 tablet_bootstrap.cc:492] T 42df1435ae4745df9f91aab1e7568270 P 13a904a6ad6a46a78dd222ee22eaea3c: Bootstrap complete.
I20260811 02:54:45.187501 29737 ts_tablet_manager.cc:1397] T 42df1435ae4745df9f91aab1e7568270 P 13a904a6ad6a46a78dd222ee22eaea3c: Time spent bootstrapping tablet: real 0.030s user 0.020s sys 0.007s
I20260811 02:54:45.188762 29863 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:54:45.171169 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.138.254:45409 (local address 127.12.138.193:40683)
0811 02:54:45.171766 (+ 597us) negotiation.cc:107] Waiting for socket to connect
0811 02:54:45.171822 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0811 02:54:45.172709 (+ 887us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:54:45.174136 (+ 1427us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:54:45.174143 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:54:45.174515 (+ 372us) client_negotiation.cc:189] Negotiated authn=SASL
0811 02:54:45.175066 (+ 551us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:54:45.175084 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:54:45.176579 (+ 1495us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:54:45.176582 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:54:45.177874 (+ 1292us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:54:45.177899 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:54:45.178913 (+ 1014us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:54:45.178916 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:54:45.179067 (+ 151us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:54:45.179708 (+ 641us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 02:54:45.179726 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0811 02:54:45.181697 (+ 1971us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 02:54:45.184787 (+ 3090us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:54:45.184793 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:54:45.184807 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:54:45.185192 (+ 385us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:54:45.185678 (+ 486us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:54:45.185683 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:54:45.185685 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:54:45.185885 (+ 200us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:54:45.186718 (+ 833us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 02:54:45.186726 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 02:54:45.187043 (+ 317us) client_negotiation.cc:769] Sending connection context
0811 02:54:45.187245 (+ 202us) client_negotiation.cc:240] Negotiation successful
0811 02:54:45.187833 (+ 588us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":388,"thread_start_us":148,"threads_started":1}
I20260811 02:54:45.188789 29862 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:54:45.171161 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.138.193:40683 (local address 127.12.138.254:45409)
0811 02:54:45.171684 (+ 523us) server_negotiation.cc:206] Beginning negotiation
0811 02:54:45.171711 (+ 27us) server_negotiation.cc:399] Waiting for connection header
0811 02:54:45.171953 (+ 242us) server_negotiation.cc:407] Connection header received
0811 02:54:45.173226 (+ 1273us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:54:45.173244 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:54:45.173665 (+ 421us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:54:45.174109 (+ 444us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:54:45.175249 (+ 1140us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:54:45.176408 (+ 1159us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:54:45.178035 (+ 1627us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:54:45.178745 (+ 710us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:54:45.178932 (+ 187us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:54:45.181957 (+ 3025us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:54:45.181968 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:54:45.181988 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:54:45.182028 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:54:45.184654 (+ 2626us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:54:45.185338 (+ 684us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:54:45.185347 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:54:45.185356 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:54:45.185568 (+ 212us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:54:45.186017 (+ 449us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:54:45.186023 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:54:45.186026 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:54:45.186585 (+ 559us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:54:45.186746 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0811 02:54:45.187442 (+ 696us) server_negotiation.cc:299] Negotiation successful
0811 02:54:45.187796 (+ 354us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":305,"thread_start_us":154,"threads_started":1}
I20260811 02:54:45.190557 29737 raft_consensus.cc:348] T 42df1435ae4745df9f91aab1e7568270 P 13a904a6ad6a46a78dd222ee22eaea3c [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a43edab73754e09a4b5e387d083f139" member_type: VOTER last_known_addr { host: "127.12.138.194" port: 45119 } } peers { permanent_uuid: "11f70dbb696346a491c8b80eaf7ce67f" member_type: VOTER last_known_addr { host: "127.12.138.195" port: 46749 } } peers { permanent_uuid: "13a904a6ad6a46a78dd222ee22eaea3c" member_type: VOTER last_known_addr { host: "127.12.138.193" port: 40085 } }
I20260811 02:54:45.191085 29737 raft_consensus.cc:729] T 42df1435ae4745df9f91aab1e7568270 P 13a904a6ad6a46a78dd222ee22eaea3c [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 13a904a6ad6a46a78dd222ee22eaea3c, State: Initialized, Role: FOLLOWER
I20260811 02:54:45.191596 29737 consensus_queue.cc:260] T 42df1435ae4745df9f91aab1e7568270 P 13a904a6ad6a46a78dd222ee22eaea3c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a43edab73754e09a4b5e387d083f139" member_type: VOTER last_known_addr { host: "127.12.138.194" port: 45119 } } peers { permanent_uuid: "11f70dbb696346a491c8b80eaf7ce67f" member_type: VOTER last_known_addr { host: "127.12.138.195" port: 46749 } } peers { permanent_uuid: "13a904a6ad6a46a78dd222ee22eaea3c" member_type: VOTER last_known_addr { host: "127.12.138.193" port: 40085 } }
I20260811 02:54:45.192152 29860 heartbeater.cc:347] Connected to a master server at 127.12.138.254:45409
I20260811 02:54:45.192382 29860 heartbeater.cc:464] Registering TS with master...
I20260811 02:54:45.192662 29737 ts_tablet_manager.cc:1428] T 42df1435ae4745df9f91aab1e7568270 P 13a904a6ad6a46a78dd222ee22eaea3c: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260811 02:54:45.192938 29860 heartbeater.cc:511] Master 127.12.138.254:45409 requested a full tablet report, sending...
I20260811 02:54:45.195549 29642 ts_manager.cc:194] Registered new tserver with Master: 13a904a6ad6a46a78dd222ee22eaea3c (127.12.138.193:40085)
I20260811 02:54:45.200165 29642 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.12.138.193@KRBTEST.COM'} at 127.12.138.193:40683
I20260811 02:54:45.203040 29860 heartbeater.cc:503] Master 127.12.138.254:45409 was elected leader, sending a full tablet report...
Aug 11 02:54:45 dist-test-slave-9gcw krb5kdc[28987](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416884, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.138.254@KRBTEST.COM for kudu/127.12.138.193@KRBTEST.COM
I20260811 02:54:45.210546 29870 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:54:45.199145 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45552 (local address 127.12.138.193:40085)
0811 02:54:45.199880 (+ 735us) server_negotiation.cc:206] Beginning negotiation
0811 02:54:45.199889 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 02:54:45.199906 (+ 17us) server_negotiation.cc:407] Connection header received
0811 02:54:45.199964 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:54:45.199968 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:54:45.200118 (+ 150us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:54:45.200278 (+ 160us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:54:45.201029 (+ 751us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:54:45.202190 (+ 1161us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:54:45.203603 (+ 1413us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:54:45.204137 (+ 534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:54:45.204246 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:54:45.206865 (+ 2619us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:54:45.206874 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:54:45.206893 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:54:45.206914 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:54:45.208591 (+ 1677us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:54:45.209200 (+ 609us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:54:45.209206 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:54:45.209207 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:54:45.209289 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:54:45.209585 (+ 296us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:54:45.209587 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:54:45.209588 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:54:45.209811 (+ 223us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:54:45.209913 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0811 02:54:45.210281 (+ 368us) server_negotiation.cc:299] Negotiation successful
0811 02:54:45.210419 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":650,"thread_start_us":109,"threads_started":1}
I20260811 02:54:45.210572 29869 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:54:45.199100 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.138.193:40085 (local address 127.0.0.1:45552)
0811 02:54:45.199469 (+ 369us) negotiation.cc:107] Waiting for socket to connect
0811 02:54:45.199515 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0811 02:54:45.199646 (+ 131us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:54:45.200329 (+ 683us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:54:45.200336 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:54:45.200749 (+ 413us) client_negotiation.cc:189] Negotiated authn=SASL
0811 02:54:45.200861 (+ 112us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:54:45.200868 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:54:45.202316 (+ 1448us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:54:45.202320 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:54:45.203448 (+ 1128us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:54:45.203462 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:54:45.204378 (+ 916us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:54:45.204382 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:54:45.204500 (+ 118us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:54:45.205064 (+ 564us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 02:54:45.205080 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0811 02:54:45.206723 (+ 1643us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 02:54:45.208705 (+ 1982us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:54:45.208714 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:54:45.208720 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:54:45.209027 (+ 307us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:54:45.209383 (+ 356us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:54:45.209386 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:54:45.209388 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:54:45.209487 (+ 99us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:54:45.209921 (+ 434us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 02:54:45.209927 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 02:54:45.210049 (+ 122us) client_negotiation.cc:769] Sending connection context
0811 02:54:45.210194 (+ 145us) client_negotiation.cc:240] Negotiation successful
0811 02:54:45.210389 (+ 195us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":188,"thread_start_us":103,"threads_started":1}
I20260811 02:54:45.212126 29787 tablet_service.cc:1514] Processing DeleteTablet for tablet 42df1435ae4745df9f91aab1e7568270 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 02:54:44 UTC) from {username='kudu', principal='kudu/127.12.138.254@KRBTEST.COM'} at 127.0.0.1:45552
I20260811 02:54:45.213179 29872 tablet_replica.cc:330] stopping tablet replica
I20260811 02:54:45.213515 29872 raft_consensus.cc:2229] T 42df1435ae4745df9f91aab1e7568270 P 13a904a6ad6a46a78dd222ee22eaea3c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 02:54:45.213779 29872 raft_consensus.cc:2258] T 42df1435ae4745df9f91aab1e7568270 P 13a904a6ad6a46a78dd222ee22eaea3c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 02:54:45.214965 29872 ts_tablet_manager.cc:1905] T 42df1435ae4745df9f91aab1e7568270 P 13a904a6ad6a46a78dd222ee22eaea3c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 02:54:45.219707 29872 ts_tablet_manager.cc:1918] T 42df1435ae4745df9f91aab1e7568270 P 13a904a6ad6a46a78dd222ee22eaea3c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 02:54:45.219828 29872 log.cc:1196] T 42df1435ae4745df9f91aab1e7568270 P 13a904a6ad6a46a78dd222ee22eaea3c: Deleting WAL directory at /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786416707705494-12843-0/minicluster-data/ts-0/wal/wals/42df1435ae4745df9f91aab1e7568270
I20260811 02:54:45.220134 29872 ts_tablet_manager.cc:1939] T 42df1435ae4745df9f91aab1e7568270 P 13a904a6ad6a46a78dd222ee22eaea3c: Deleting consensus metadata
I20260811 02:54:45.222332 29634 catalog_manager.cc:4841] TS 13a904a6ad6a46a78dd222ee22eaea3c (127.12.138.193:40085): tablet 42df1435ae4745df9f91aab1e7568270 (table test-table [id=a8015fa6fe2842dc8ecc81a639578020]) successfully deleted
W20260811 02:54:45.308512 29865 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 02:54:45.308876 29865 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 02:54:45.308974 29865 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 02:54:45.313879 29865 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 02:54:45.313994 29865 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 02:54:45.314101 29865 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.138.194
I20260811 02:54:45.320360 29865 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.138.212:45835
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786416707705494-12843-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786416707705494-12843-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786416707705494-12843-0/krb5kdc/kudu_127.12.138.194.keytab
--principal=kudu/127.12.138.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.138.194:45119
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786416707705494-12843-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.12.138.194
--webserver_port=38055
--tserver_master_addrs=127.12.138.254:45409
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29865
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.138.194
--log_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786416707705494-12843-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 02:09:49 UTC on ccycloud.quasar-fsnqqc.root.comops.site
build id 4579570
I20260811 02:54:45.321779 29865 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 02:54:45.323015 29865 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 02:54:45.333317 29882 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:54:45.333326 29879 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:54:45.333487 29865 server_base.cc:1034] running on GCE node
W20260811 02:54:45.333505 29880 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:54:45.334100 29865 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 02:54:45.334725 29865 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 02:54:45.335911 29865 hybrid_clock.cc:648] HybridClock initialized: now 1786416885335879 us; error 42 us; skew 500 ppm
Aug 11 02:54:45 dist-test-slave-9gcw krb5kdc[28987](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416885, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.138.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 02:54:45.339020 29865 init.cc:377] Logged in from keytab as kudu/127.12.138.194@KRBTEST.COM (short username kudu)
I20260811 02:54:45.339987 29865 webserver.cc:460] Webserver started at http://127.12.138.194:38055/ using document root <none> and password file <none>
I20260811 02:54:45.340597 29865 fs_manager.cc:359] Metadata directory not provided
I20260811 02:54:45.340700 29865 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 02:54:45.344733 29865 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.000s
I20260811 02:54:45.347021 29889 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 02:54:45.348294 29865 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260811 02:54:45.348416 29865 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786416707705494-12843-0/minicluster-data/ts-1/data,/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786416707705494-12843-0/minicluster-data/ts-1/wal
uuid: "8a43edab73754e09a4b5e387d083f139"
format_stamp: "Formatted at 2026-08-11 02:54:44 on dist-test-slave-9gcw"
server_key: "0b24223102881f0eeb80a43fe932ab86"
server_key_iv: "1d7ad0a13c8dd59c653d43d0d1ab63ac"
server_key_version: "encryptionkey@0"
I20260811 02:54:45.348969 29865 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786416707705494-12843-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786416707705494-12843-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786416707705494-12843-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 02:54:45.368683 29865 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 02:54:45.372190 29865 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 02:54:45.372424 29865 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 02:54:45.373087 29865 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 02:54:45.374286 29899 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260811 02:54:45.379283 29865 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260811 02:54:45.379360 29865 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.005s user 0.000s sys 0.000s
I20260811 02:54:45.379454 29865 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260811 02:54:45.381289 29865 ts_tablet_manager.cc:610] Registered 1 tablets
I20260811 02:54:45.381340 29865 ts_tablet_manager.cc:589] Time spent register tablets: real 0.002s user 0.002s sys 0.000s
I20260811 02:54:45.381731 29899 tablet_bootstrap.cc:492] T 42df1435ae4745df9f91aab1e7568270 P 8a43edab73754e09a4b5e387d083f139: Bootstrap starting.
I20260811 02:54:45.389290 29899 log.cc:824] T 42df1435ae4745df9f91aab1e7568270 P 8a43edab73754e09a4b5e387d083f139: Log is configured to *not* fsync() on all Append() calls
I20260811 02:54:45.393940 29865 rpc_server.cc:307] RPC server started. Bound to: 127.12.138.194:45119
I20260811 02:54:45.394038 30022 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.138.194:45119 every 8 connection(s)
I20260811 02:54:45.394970 29865 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786416707705494-12843-0/minicluster-data/ts-1/data/info.pb
I20260811 02:54:45.396526 12843 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDawEwS/build/debug/bin/kudu as pid 29865
I20260811 02:54:45.397210 12843 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDawEwS/build/debug/bin/kudu
/tmp/dist-test-taskDawEwS/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786416707705494-12843-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786416707705494-12843-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786416707705494-12843-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786416707705494-12843-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.138.195:46749
--local_ip_for_outbound_sockets=127.12.138.195
--tserver_master_addrs=127.12.138.254:45409
--webserver_port=42981
--webserver_interface=127.12.138.195
--keytab_file=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786416707705494-12843-0/krb5kdc/kudu_127.12.138.195.keytab
--principal=kudu/127.12.138.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.138.212:45835
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786416707705494-12843-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786416707705494-12843-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786416707705494-12843-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 02:54:45.399850 29899 tablet_bootstrap.cc:492] T 42df1435ae4745df9f91aab1e7568270 P 8a43edab73754e09a4b5e387d083f139: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 02:54:45.400476 29899 tablet_bootstrap.cc:492] T 42df1435ae4745df9f91aab1e7568270 P 8a43edab73754e09a4b5e387d083f139: Bootstrap complete.
I20260811 02:54:45.400956 29899 ts_tablet_manager.cc:1397] T 42df1435ae4745df9f91aab1e7568270 P 8a43edab73754e09a4b5e387d083f139: Time spent bootstrapping tablet: real 0.019s user 0.013s sys 0.005s
I20260811 02:54:45.403908 29899 raft_consensus.cc:348] T 42df1435ae4745df9f91aab1e7568270 P 8a43edab73754e09a4b5e387d083f139 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a43edab73754e09a4b5e387d083f139" member_type: VOTER last_known_addr { host: "127.12.138.194" port: 45119 } } peers { permanent_uuid: "11f70dbb696346a491c8b80eaf7ce67f" member_type: VOTER last_known_addr { host: "127.12.138.195" port: 46749 } } peers { permanent_uuid: "13a904a6ad6a46a78dd222ee22eaea3c" member_type: VOTER last_known_addr { host: "127.12.138.193" port: 40085 } }
I20260811 02:54:45.404429 29899 raft_consensus.cc:729] T 42df1435ae4745df9f91aab1e7568270 P 8a43edab73754e09a4b5e387d083f139 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8a43edab73754e09a4b5e387d083f139, State: Initialized, Role: FOLLOWER
I20260811 02:54:45.404954 29899 consensus_queue.cc:260] T 42df1435ae4745df9f91aab1e7568270 P 8a43edab73754e09a4b5e387d083f139 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a43edab73754e09a4b5e387d083f139" member_type: VOTER last_known_addr { host: "127.12.138.194" port: 45119 } } peers { permanent_uuid: "11f70dbb696346a491c8b80eaf7ce67f" member_type: VOTER last_known_addr { host: "127.12.138.195" port: 46749 } } peers { permanent_uuid: "13a904a6ad6a46a78dd222ee22eaea3c" member_type: VOTER last_known_addr { host: "127.12.138.193" port: 40085 } }
I20260811 02:54:45.406066 29899 ts_tablet_manager.cc:1428] T 42df1435ae4745df9f91aab1e7568270 P 8a43edab73754e09a4b5e387d083f139: Time spent starting tablet: real 0.005s user 0.004s sys 0.001s
Aug 11 02:54:45 dist-test-slave-9gcw krb5kdc[28987](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416885, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.138.194@KRBTEST.COM for kudu/127.12.138.254@KRBTEST.COM
I20260811 02:54:45.413532 29862 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:54:45.397076 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.138.194:38469 (local address 127.12.138.254:45409)
0811 02:54:45.397260 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0811 02:54:45.397266 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 02:54:45.397984 (+ 718us) server_negotiation.cc:407] Connection header received
0811 02:54:45.399237 (+ 1253us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:54:45.399242 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:54:45.399303 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:54:45.399366 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:54:45.401442 (+ 2076us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:54:45.402722 (+ 1280us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:54:45.404449 (+ 1727us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:54:45.404968 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:54:45.405130 (+ 162us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:54:45.408664 (+ 3534us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:54:45.408672 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:54:45.408677 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:54:45.408720 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:54:45.410962 (+ 2242us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:54:45.411626 (+ 664us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:54:45.411630 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:54:45.411632 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:54:45.411756 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:54:45.412179 (+ 423us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:54:45.412185 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:54:45.412190 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:54:45.412522 (+ 332us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:54:45.412667 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0811 02:54:45.413179 (+ 512us) server_negotiation.cc:299] Negotiation successful
0811 02:54:45.413344 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260811 02:54:45.414310 30026 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:54:45.397283 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.138.254:45409 (local address 127.12.138.194:38469)
0811 02:54:45.397817 (+ 534us) negotiation.cc:107] Waiting for socket to connect
0811 02:54:45.397870 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0811 02:54:45.398966 (+ 1096us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:54:45.399850 (+ 884us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:54:45.399861 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:54:45.400527 (+ 666us) client_negotiation.cc:189] Negotiated authn=SASL
0811 02:54:45.401252 (+ 725us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:54:45.401273 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:54:45.402921 (+ 1648us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:54:45.402925 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:54:45.404294 (+ 1369us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:54:45.404304 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:54:45.405180 (+ 876us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:54:45.405188 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:54:45.405388 (+ 200us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:54:45.406606 (+ 1218us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 02:54:45.406624 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0811 02:54:45.408450 (+ 1826us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 02:54:45.411198 (+ 2748us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:54:45.411207 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:54:45.411222 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:54:45.411491 (+ 269us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:54:45.411873 (+ 382us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:54:45.411876 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:54:45.411877 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:54:45.412053 (+ 176us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:54:45.412652 (+ 599us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 02:54:45.412658 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 02:54:45.412894 (+ 236us) client_negotiation.cc:769] Sending connection context
0811 02:54:45.413046 (+ 152us) client_negotiation.cc:240] Negotiation successful
0811 02:54:45.413389 (+ 343us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":315,"thread_start_us":145,"threads_started":1}
I20260811 02:54:45.415549 30024 heartbeater.cc:347] Connected to a master server at 127.12.138.254:45409
I20260811 02:54:45.415834 30024 heartbeater.cc:464] Registering TS with master...
I20260811 02:54:45.416369 30024 heartbeater.cc:511] Master 127.12.138.254:45409 requested a full tablet report, sending...
I20260811 02:54:45.418658 29642 ts_manager.cc:194] Registered new tserver with Master: 8a43edab73754e09a4b5e387d083f139 (127.12.138.194:45119)
I20260811 02:54:45.419977 29642 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.12.138.194@KRBTEST.COM'} at 127.12.138.194:38469
I20260811 02:54:45.422706 30024 heartbeater.cc:503] Master 127.12.138.254:45409 was elected leader, sending a full tablet report...
Aug 11 02:54:45 dist-test-slave-9gcw krb5kdc[28987](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416884, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.138.254@KRBTEST.COM for kudu/127.12.138.194@KRBTEST.COM
I20260811 02:54:45.429960 29869 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:54:45.419583 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.138.194:45119 (local address 127.0.0.1:39906)
0811 02:54:45.419769 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0811 02:54:45.419783 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 02:54:45.419872 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:54:45.420586 (+ 714us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:54:45.420589 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:54:45.420799 (+ 210us) client_negotiation.cc:189] Negotiated authn=SASL
0811 02:54:45.420921 (+ 122us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:54:45.420929 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:54:45.422302 (+ 1373us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:54:45.422306 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:54:45.422924 (+ 618us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:54:45.422934 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:54:45.423725 (+ 791us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:54:45.423730 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:54:45.423866 (+ 136us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:54:45.424492 (+ 626us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 02:54:45.424509 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 02:54:45.426038 (+ 1529us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 02:54:45.428341 (+ 2303us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:54:45.428343 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:54:45.428345 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:54:45.428555 (+ 210us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:54:45.428900 (+ 345us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:54:45.428903 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:54:45.428904 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:54:45.428981 (+ 77us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:54:45.429542 (+ 561us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 02:54:45.429544 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 02:54:45.429629 (+ 85us) client_negotiation.cc:769] Sending connection context
0811 02:54:45.429713 (+ 84us) client_negotiation.cc:240] Negotiation successful
0811 02:54:45.429837 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":116}
I20260811 02:54:45.430205 30030 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:54:45.419753 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39906 (local address 127.12.138.194:45119)
0811 02:54:45.420127 (+ 374us) server_negotiation.cc:206] Beginning negotiation
0811 02:54:45.420139 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 02:54:45.420162 (+ 23us) server_negotiation.cc:407] Connection header received
0811 02:54:45.420233 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:54:45.420239 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:54:45.420468 (+ 229us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:54:45.420605 (+ 137us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:54:45.421038 (+ 433us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:54:45.422185 (+ 1147us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:54:45.423036 (+ 851us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:54:45.423573 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:54:45.423700 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:54:45.426190 (+ 2490us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:54:45.426200 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:54:45.426216 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:54:45.426243 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:54:45.428220 (+ 1977us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:54:45.428678 (+ 458us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:54:45.428684 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:54:45.428686 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:54:45.428793 (+ 107us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:54:45.429072 (+ 279us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:54:45.429076 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:54:45.429079 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:54:45.429433 (+ 354us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:54:45.429580 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0811 02:54:45.429831 (+ 251us) server_negotiation.cc:299] Negotiation successful
0811 02:54:45.430017 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":51,"server-negotiator.queue_time_us":274,"thread_start_us":113,"threads_started":1}
I20260811 02:54:45.431162 29946 tablet_service.cc:1514] Processing DeleteTablet for tablet 42df1435ae4745df9f91aab1e7568270 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 02:54:44 UTC) from {username='kudu', principal='kudu/127.12.138.254@KRBTEST.COM'} at 127.0.0.1:39906
I20260811 02:54:45.432106 30031 tablet_replica.cc:330] stopping tablet replica
I20260811 02:54:45.432401 30031 raft_consensus.cc:2229] T 42df1435ae4745df9f91aab1e7568270 P 8a43edab73754e09a4b5e387d083f139 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 02:54:45.432655 30031 raft_consensus.cc:2258] T 42df1435ae4745df9f91aab1e7568270 P 8a43edab73754e09a4b5e387d083f139 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 02:54:45.433672 30031 ts_tablet_manager.cc:1905] T 42df1435ae4745df9f91aab1e7568270 P 8a43edab73754e09a4b5e387d083f139: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 02:54:45.437366 30031 ts_tablet_manager.cc:1918] T 42df1435ae4745df9f91aab1e7568270 P 8a43edab73754e09a4b5e387d083f139: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 02:54:45.437450 30031 log.cc:1196] T 42df1435ae4745df9f91aab1e7568270 P 8a43edab73754e09a4b5e387d083f139: Deleting WAL directory at /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786416707705494-12843-0/minicluster-data/ts-1/wal/wals/42df1435ae4745df9f91aab1e7568270
I20260811 02:54:45.437719 30031 ts_tablet_manager.cc:1939] T 42df1435ae4745df9f91aab1e7568270 P 8a43edab73754e09a4b5e387d083f139: Deleting consensus metadata
I20260811 02:54:45.439662 29634 catalog_manager.cc:4841] TS 8a43edab73754e09a4b5e387d083f139 (127.12.138.194:45119): tablet 42df1435ae4745df9f91aab1e7568270 (table test-table [id=a8015fa6fe2842dc8ecc81a639578020]) successfully deleted
W20260811 02:54:45.534626 30027 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 02:54:45.534937 30027 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 02:54:45.535023 30027 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 02:54:45.539075 30027 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 02:54:45.539183 30027 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 02:54:45.539287 30027 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.138.195
I20260811 02:54:45.544611 30027 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.138.212:45835
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786416707705494-12843-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786416707705494-12843-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786416707705494-12843-0/krb5kdc/kudu_127.12.138.195.keytab
--principal=kudu/127.12.138.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.138.195:46749
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786416707705494-12843-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.12.138.195
--webserver_port=42981
--tserver_master_addrs=127.12.138.254:45409
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30027
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.138.195
--log_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786416707705494-12843-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 02:09:49 UTC on ccycloud.quasar-fsnqqc.root.comops.site
build id 4579570
I20260811 02:54:45.545773 30027 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 02:54:45.546841 30027 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 02:54:45.557139 30037 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:54:45.557139 30040 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:54:45.557176 30036 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:54:45.557498 30027 server_base.cc:1034] running on GCE node
I20260811 02:54:45.557981 30027 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 02:54:45.558671 30027 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 02:54:45.559897 30027 hybrid_clock.cc:648] HybridClock initialized: now 1786416885559861 us; error 46 us; skew 500 ppm
Aug 11 02:54:45 dist-test-slave-9gcw krb5kdc[28987](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416885, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.138.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 02:54:45.562824 30027 init.cc:377] Logged in from keytab as kudu/127.12.138.195@KRBTEST.COM (short username kudu)
I20260811 02:54:45.563836 30027 webserver.cc:460] Webserver started at http://127.12.138.195:42981/ using document root <none> and password file <none>
I20260811 02:54:45.564500 30027 fs_manager.cc:359] Metadata directory not provided
I20260811 02:54:45.564618 30027 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 02:54:45.568637 30027 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.000s
I20260811 02:54:45.570796 30049 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 02:54:45.572075 30027 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260811 02:54:45.572271 30027 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786416707705494-12843-0/minicluster-data/ts-2/data,/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786416707705494-12843-0/minicluster-data/ts-2/wal
uuid: "11f70dbb696346a491c8b80eaf7ce67f"
format_stamp: "Formatted at 2026-08-11 02:54:44 on dist-test-slave-9gcw"
server_key: "0eacae20ba2b9cf328c5a1f43f4d883a"
server_key_iv: "85600423a907dc8209ef1b09bcff9144"
server_key_version: "encryptionkey@0"
I20260811 02:54:45.572901 30027 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786416707705494-12843-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786416707705494-12843-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786416707705494-12843-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 02:54:45.603914 30027 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 02:54:45.607072 30027 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 02:54:45.607313 30027 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 02:54:45.607949 30027 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 02:54:45.608848 30027 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 02:54:45.608922 30027 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 02:54:45.608999 30027 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 02:54:45.609050 30027 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 02:54:45.619127 30027 rpc_server.cc:307] RPC server started. Bound to: 127.12.138.195:46749
I20260811 02:54:45.619150 30178 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.138.195:46749 every 8 connection(s)
I20260811 02:54:45.620232 30027 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786416707705494-12843-0/minicluster-data/ts-2/data/info.pb
I20260811 02:54:45.627924 12843 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDawEwS/build/debug/bin/kudu as pid 30027
Aug 11 02:54:45 dist-test-slave-9gcw krb5kdc[28987](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416885, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.138.195@KRBTEST.COM for kudu/127.12.138.254@KRBTEST.COM
I20260811 02:54:45.636832 29862 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:54:45.622394 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.138.195:41853 (local address 127.12.138.254:45409)
0811 02:54:45.622575 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0811 02:54:45.622580 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 02:54:45.623409 (+ 829us) server_negotiation.cc:407] Connection header received
0811 02:54:45.625046 (+ 1637us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:54:45.625051 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:54:45.625115 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:54:45.625244 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:54:45.626556 (+ 1312us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:54:45.627169 (+ 613us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:54:45.628587 (+ 1418us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:54:45.628961 (+ 374us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:54:45.629097 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:54:45.632363 (+ 3266us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:54:45.632366 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:54:45.632368 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:54:45.632394 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:54:45.634185 (+ 1791us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:54:45.634883 (+ 698us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:54:45.634885 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:54:45.634886 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:54:45.634968 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:54:45.635456 (+ 488us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:54:45.635459 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:54:45.635461 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:54:45.635728 (+ 267us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:54:45.635818 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0811 02:54:45.636561 (+ 743us) server_negotiation.cc:299] Negotiation successful
0811 02:54:45.636705 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260811 02:54:45.638019 30185 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:54:45.628712 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58640 (local address 127.12.138.254:45409)
0811 02:54:45.629354 (+ 642us) server_negotiation.cc:206] Beginning negotiation
0811 02:54:45.629358 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 02:54:45.629370 (+ 12us) server_negotiation.cc:407] Connection header received
0811 02:54:45.629404 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:54:45.629406 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:54:45.629444 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:54:45.629550 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:54:45.630487 (+ 937us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:54:45.631122 (+ 635us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:54:45.632177 (+ 1055us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:54:45.632501 (+ 324us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:54:45.632589 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:54:45.634198 (+ 1609us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:54:45.634201 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:54:45.634203 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:54:45.634228 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:54:45.636122 (+ 1894us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:54:45.636655 (+ 533us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:54:45.636659 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:54:45.636660 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:54:45.636774 (+ 114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:54:45.637126 (+ 352us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:54:45.637129 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:54:45.637138 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:54:45.637371 (+ 233us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:54:45.637517 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0811 02:54:45.637787 (+ 270us) server_negotiation.cc:299] Negotiation successful
0811 02:54:45.637884 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":556,"thread_start_us":104,"threads_started":1}
I20260811 02:54:45.638096 30183 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:54:45.622748 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.138.254:45409 (local address 127.12.138.195:41853)
0811 02:54:45.623237 (+ 489us) negotiation.cc:107] Waiting for socket to connect
0811 02:54:45.623284 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0811 02:54:45.624789 (+ 1505us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:54:45.625433 (+ 644us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:54:45.625440 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:54:45.625857 (+ 417us) client_negotiation.cc:189] Negotiated authn=SASL
0811 02:54:45.626384 (+ 527us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:54:45.626398 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:54:45.627324 (+ 926us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:54:45.627328 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:54:45.628440 (+ 1112us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:54:45.628451 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:54:45.629170 (+ 719us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:54:45.629176 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:54:45.629369 (+ 193us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:54:45.630237 (+ 868us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 02:54:45.630256 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0811 02:54:45.632169 (+ 1913us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 02:54:45.634309 (+ 2140us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:54:45.634318 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:54:45.634337 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:54:45.634758 (+ 421us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:54:45.635071 (+ 313us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:54:45.635078 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:54:45.635084 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:54:45.635311 (+ 227us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:54:45.635858 (+ 547us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 02:54:45.635868 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 02:54:45.636250 (+ 382us) client_negotiation.cc:769] Sending connection context
0811 02:54:45.636465 (+ 215us) client_negotiation.cc:240] Negotiation successful
0811 02:54:45.636856 (+ 391us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":290,"thread_start_us":124,"threads_started":1}
I20260811 02:54:45.639359 30181 heartbeater.cc:347] Connected to a master server at 127.12.138.254:45409
I20260811 02:54:45.639613 30181 heartbeater.cc:464] Registering TS with master...
I20260811 02:54:45.640228 30181 heartbeater.cc:511] Master 127.12.138.254:45409 requested a full tablet report, sending...
I20260811 02:54:45.641302 29642 ts_manager.cc:194] Registered new tserver with Master: 11f70dbb696346a491c8b80eaf7ce67f (127.12.138.195:46749)
I20260811 02:54:45.641819 29642 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.12.138.195@KRBTEST.COM'} at 127.12.138.195:41853
I20260811 02:54:45.651970 12843 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 02:54:45.656939 30185 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:54:45.652641 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58642 (local address 127.12.138.254:45409)
0811 02:54:45.652832 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0811 02:54:45.652836 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 02:54:45.653102 (+ 266us) server_negotiation.cc:407] Connection header received
0811 02:54:45.653353 (+ 251us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:54:45.653356 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:54:45.653466 (+ 110us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:54:45.653586 (+ 120us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 02:54:45.653900 (+ 314us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:54:45.654451 (+ 551us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:54:45.655413 (+ 962us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:54:45.655659 (+ 246us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:54:45.655770 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:54:45.656109 (+ 339us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 02:54:45.656532 (+ 423us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 02:54:45.656669 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0811 02:54:45.656733 (+ 64us) server_negotiation.cc:299] Negotiation successful
0811 02:54:45.656790 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260811 02:54:45.657671 29642 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin'} at 127.0.0.1:58642:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 02:54:45.661024 29642 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 02:54:45.668644 29946 tablet_service.cc:1467] Processing CreateTablet for tablet 89669c29a2d34fa98ad0180c930e32dd (DEFAULT_TABLE table=test-table [id=b338f0c35cdd49408005b6579baff66b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 02:54:45.668704 29787 tablet_service.cc:1467] Processing CreateTablet for tablet 89669c29a2d34fa98ad0180c930e32dd (DEFAULT_TABLE table=test-table [id=b338f0c35cdd49408005b6579baff66b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 02:54:45.669349 29946 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 89669c29a2d34fa98ad0180c930e32dd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 02:54:45.669351 29787 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 89669c29a2d34fa98ad0180c930e32dd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 02:54:45.672117 29899 tablet_bootstrap.cc:492] T 89669c29a2d34fa98ad0180c930e32dd P 8a43edab73754e09a4b5e387d083f139: Bootstrap starting.
I20260811 02:54:45.672232 29737 tablet_bootstrap.cc:492] T 89669c29a2d34fa98ad0180c930e32dd P 13a904a6ad6a46a78dd222ee22eaea3c: Bootstrap starting.
I20260811 02:54:45.673065 29899 tablet_bootstrap.cc:654] T 89669c29a2d34fa98ad0180c930e32dd P 8a43edab73754e09a4b5e387d083f139: Neither blocks nor log segments found. Creating new log.
I20260811 02:54:45.673069 29737 tablet_bootstrap.cc:654] T 89669c29a2d34fa98ad0180c930e32dd P 13a904a6ad6a46a78dd222ee22eaea3c: Neither blocks nor log segments found. Creating new log.
I20260811 02:54:45.673883 29869 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:54:45.667768 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.138.195:46749 (local address 127.0.0.1:44732)
0811 02:54:45.668109 (+ 341us) negotiation.cc:107] Waiting for socket to connect
0811 02:54:45.668142 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0811 02:54:45.668226 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:54:45.669265 (+ 1039us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:54:45.669268 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:54:45.669288 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 02:54:45.669401 (+ 113us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:54:45.669408 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:54:45.670856 (+ 1448us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:54:45.670860 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:54:45.672076 (+ 1216us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:54:45.672087 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:54:45.673368 (+ 1281us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:54:45.673373 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:54:45.673497 (+ 124us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:54:45.673555 (+ 58us) client_negotiation.cc:769] Sending connection context
0811 02:54:45.673649 (+ 94us) client_negotiation.cc:240] Negotiation successful
0811 02:54:45.673737 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":257}
I20260811 02:54:45.673979 30192 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:54:45.668296 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44732 (local address 127.12.138.195:46749)
0811 02:54:45.668730 (+ 434us) server_negotiation.cc:206] Beginning negotiation
0811 02:54:45.668744 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0811 02:54:45.668769 (+ 25us) server_negotiation.cc:407] Connection header received
0811 02:54:45.668853 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:54:45.668858 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:54:45.669078 (+ 220us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:54:45.669274 (+ 196us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 02:54:45.669552 (+ 278us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:54:45.670691 (+ 1139us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:54:45.672241 (+ 1550us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:54:45.673107 (+ 866us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:54:45.673278 (+ 171us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:54:45.673441 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0811 02:54:45.673712 (+ 271us) server_negotiation.cc:299] Negotiation successful
0811 02:54:45.673817 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":316,"thread_start_us":197,"threads_started":1}
I20260811 02:54:45.674628 29899 tablet_bootstrap.cc:492] T 89669c29a2d34fa98ad0180c930e32dd P 8a43edab73754e09a4b5e387d083f139: No bootstrap required, opened a new log
I20260811 02:54:45.674732 29899 ts_tablet_manager.cc:1397] T 89669c29a2d34fa98ad0180c930e32dd P 8a43edab73754e09a4b5e387d083f139: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.002s
I20260811 02:54:45.674767 29737 tablet_bootstrap.cc:492] T 89669c29a2d34fa98ad0180c930e32dd P 13a904a6ad6a46a78dd222ee22eaea3c: No bootstrap required, opened a new log
I20260811 02:54:45.674858 29737 ts_tablet_manager.cc:1397] T 89669c29a2d34fa98ad0180c930e32dd P 13a904a6ad6a46a78dd222ee22eaea3c: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260811 02:54:45.675189 29899 raft_consensus.cc:348] T 89669c29a2d34fa98ad0180c930e32dd P 8a43edab73754e09a4b5e387d083f139 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "11f70dbb696346a491c8b80eaf7ce67f" member_type: VOTER last_known_addr { host: "127.12.138.195" port: 46749 } } peers { permanent_uuid: "8a43edab73754e09a4b5e387d083f139" member_type: VOTER last_known_addr { host: "127.12.138.194" port: 45119 } } peers { permanent_uuid: "13a904a6ad6a46a78dd222ee22eaea3c" member_type: VOTER last_known_addr { host: "127.12.138.193" port: 40085 } }
I20260811 02:54:45.675323 29899 raft_consensus.cc:374] T 89669c29a2d34fa98ad0180c930e32dd P 8a43edab73754e09a4b5e387d083f139 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 02:54:45.675292 29737 raft_consensus.cc:348] T 89669c29a2d34fa98ad0180c930e32dd P 13a904a6ad6a46a78dd222ee22eaea3c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "11f70dbb696346a491c8b80eaf7ce67f" member_type: VOTER last_known_addr { host: "127.12.138.195" port: 46749 } } peers { permanent_uuid: "8a43edab73754e09a4b5e387d083f139" member_type: VOTER last_known_addr { host: "127.12.138.194" port: 45119 } } peers { permanent_uuid: "13a904a6ad6a46a78dd222ee22eaea3c" member_type: VOTER last_known_addr { host: "127.12.138.193" port: 40085 } }
I20260811 02:54:45.675383 29899 raft_consensus.cc:729] T 89669c29a2d34fa98ad0180c930e32dd P 8a43edab73754e09a4b5e387d083f139 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8a43edab73754e09a4b5e387d083f139, State: Initialized, Role: FOLLOWER
I20260811 02:54:45.675418 29737 raft_consensus.cc:374] T 89669c29a2d34fa98ad0180c930e32dd P 13a904a6ad6a46a78dd222ee22eaea3c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 02:54:45.675465 29737 raft_consensus.cc:729] T 89669c29a2d34fa98ad0180c930e32dd P 13a904a6ad6a46a78dd222ee22eaea3c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 13a904a6ad6a46a78dd222ee22eaea3c, State: Initialized, Role: FOLLOWER
I20260811 02:54:45.675513 29899 consensus_queue.cc:260] T 89669c29a2d34fa98ad0180c930e32dd P 8a43edab73754e09a4b5e387d083f139 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "11f70dbb696346a491c8b80eaf7ce67f" member_type: VOTER last_known_addr { host: "127.12.138.195" port: 46749 } } peers { permanent_uuid: "8a43edab73754e09a4b5e387d083f139" member_type: VOTER last_known_addr { host: "127.12.138.194" port: 45119 } } peers { permanent_uuid: "13a904a6ad6a46a78dd222ee22eaea3c" member_type: VOTER last_known_addr { host: "127.12.138.193" port: 40085 } }
I20260811 02:54:45.675577 29737 consensus_queue.cc:260] T 89669c29a2d34fa98ad0180c930e32dd P 13a904a6ad6a46a78dd222ee22eaea3c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "11f70dbb696346a491c8b80eaf7ce67f" member_type: VOTER last_known_addr { host: "127.12.138.195" port: 46749 } } peers { permanent_uuid: "8a43edab73754e09a4b5e387d083f139" member_type: VOTER last_known_addr { host: "127.12.138.194" port: 45119 } } peers { permanent_uuid: "13a904a6ad6a46a78dd222ee22eaea3c" member_type: VOTER last_known_addr { host: "127.12.138.193" port: 40085 } }
I20260811 02:54:45.675807 29737 ts_tablet_manager.cc:1428] T 89669c29a2d34fa98ad0180c930e32dd P 13a904a6ad6a46a78dd222ee22eaea3c: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260811 02:54:45.675804 29899 ts_tablet_manager.cc:1428] T 89669c29a2d34fa98ad0180c930e32dd P 8a43edab73754e09a4b5e387d083f139: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260811 02:54:45.677428 30101 tablet_service.cc:1467] Processing CreateTablet for tablet 89669c29a2d34fa98ad0180c930e32dd (DEFAULT_TABLE table=test-table [id=b338f0c35cdd49408005b6579baff66b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 02:54:45.678561 30101 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 89669c29a2d34fa98ad0180c930e32dd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 02:54:45.685541 30195 tablet_bootstrap.cc:492] T 89669c29a2d34fa98ad0180c930e32dd P 11f70dbb696346a491c8b80eaf7ce67f: Bootstrap starting.
I20260811 02:54:45.688277 30195 tablet_bootstrap.cc:654] T 89669c29a2d34fa98ad0180c930e32dd P 11f70dbb696346a491c8b80eaf7ce67f: Neither blocks nor log segments found. Creating new log.
I20260811 02:54:45.689183 30195 log.cc:824] T 89669c29a2d34fa98ad0180c930e32dd P 11f70dbb696346a491c8b80eaf7ce67f: Log is configured to *not* fsync() on all Append() calls
I20260811 02:54:45.691181 30195 tablet_bootstrap.cc:492] T 89669c29a2d34fa98ad0180c930e32dd P 11f70dbb696346a491c8b80eaf7ce67f: No bootstrap required, opened a new log
I20260811 02:54:45.691401 30195 ts_tablet_manager.cc:1397] T 89669c29a2d34fa98ad0180c930e32dd P 11f70dbb696346a491c8b80eaf7ce67f: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260811 02:54:45.694447 30195 raft_consensus.cc:348] T 89669c29a2d34fa98ad0180c930e32dd P 11f70dbb696346a491c8b80eaf7ce67f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "11f70dbb696346a491c8b80eaf7ce67f" member_type: VOTER last_known_addr { host: "127.12.138.195" port: 46749 } } peers { permanent_uuid: "8a43edab73754e09a4b5e387d083f139" member_type: VOTER last_known_addr { host: "127.12.138.194" port: 45119 } } peers { permanent_uuid: "13a904a6ad6a46a78dd222ee22eaea3c" member_type: VOTER last_known_addr { host: "127.12.138.193" port: 40085 } }
I20260811 02:54:45.694662 30195 raft_consensus.cc:374] T 89669c29a2d34fa98ad0180c930e32dd P 11f70dbb696346a491c8b80eaf7ce67f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 02:54:45.694743 30195 raft_consensus.cc:729] T 89669c29a2d34fa98ad0180c930e32dd P 11f70dbb696346a491c8b80eaf7ce67f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 11f70dbb696346a491c8b80eaf7ce67f, State: Initialized, Role: FOLLOWER
I20260811 02:54:45.695164 30195 consensus_queue.cc:260] T 89669c29a2d34fa98ad0180c930e32dd P 11f70dbb696346a491c8b80eaf7ce67f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "11f70dbb696346a491c8b80eaf7ce67f" member_type: VOTER last_known_addr { host: "127.12.138.195" port: 46749 } } peers { permanent_uuid: "8a43edab73754e09a4b5e387d083f139" member_type: VOTER last_known_addr { host: "127.12.138.194" port: 45119 } } peers { permanent_uuid: "13a904a6ad6a46a78dd222ee22eaea3c" member_type: VOTER last_known_addr { host: "127.12.138.193" port: 40085 } }
I20260811 02:54:45.695884 30181 heartbeater.cc:503] Master 127.12.138.254:45409 was elected leader, sending a full tablet report...
I20260811 02:54:45.696069 30195 ts_tablet_manager.cc:1428] T 89669c29a2d34fa98ad0180c930e32dd P 11f70dbb696346a491c8b80eaf7ce67f: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
W20260811 02:54:45.871443 30182 tablet.cc:2369] T 89669c29a2d34fa98ad0180c930e32dd P 11f70dbb696346a491c8b80eaf7ce67f: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 02:54:45.889773 29867 raft_consensus.cc:482] T 89669c29a2d34fa98ad0180c930e32dd P 13a904a6ad6a46a78dd222ee22eaea3c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 02:54:45.890033 29867 raft_consensus.cc:504] T 89669c29a2d34fa98ad0180c930e32dd P 13a904a6ad6a46a78dd222ee22eaea3c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "11f70dbb696346a491c8b80eaf7ce67f" member_type: VOTER last_known_addr { host: "127.12.138.195" port: 46749 } } peers { permanent_uuid: "8a43edab73754e09a4b5e387d083f139" member_type: VOTER last_known_addr { host: "127.12.138.194" port: 45119 } } peers { permanent_uuid: "13a904a6ad6a46a78dd222ee22eaea3c" member_type: VOTER last_known_addr { host: "127.12.138.193" port: 40085 } }
I20260811 02:54:45.891355 29867 leader_election.cc:288] T 89669c29a2d34fa98ad0180c930e32dd P 13a904a6ad6a46a78dd222ee22eaea3c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 11f70dbb696346a491c8b80eaf7ce67f (127.12.138.195:46749), 8a43edab73754e09a4b5e387d083f139 (127.12.138.194:45119)
I20260811 02:54:45.895754 30030 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:54:45.891799 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.138.193:46185 (local address 127.12.138.194:45119)
0811 02:54:45.891938 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0811 02:54:45.891943 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 02:54:45.892152 (+ 209us) server_negotiation.cc:407] Connection header received
0811 02:54:45.892325 (+ 173us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:54:45.892328 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:54:45.892381 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:54:45.892488 (+ 107us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 02:54:45.892751 (+ 263us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:54:45.893341 (+ 590us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:54:45.894358 (+ 1017us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:54:45.895037 (+ 679us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:54:45.895158 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:54:45.895313 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0811 02:54:45.895550 (+ 237us) server_negotiation.cc:299] Negotiation successful
0811 02:54:45.895622 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260811 02:54:45.895754 30192 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:54:45.891745 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.138.193:53999 (local address 127.12.138.195:46749)
0811 02:54:45.891905 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0811 02:54:45.891910 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 02:54:45.892087 (+ 177us) server_negotiation.cc:407] Connection header received
0811 02:54:45.892305 (+ 218us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:54:45.892313 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:54:45.892357 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:54:45.892449 (+ 92us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 02:54:45.892727 (+ 278us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:54:45.893275 (+ 548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:54:45.894427 (+ 1152us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:54:45.895029 (+ 602us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:54:45.895158 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:54:45.895214 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0811 02:54:45.895550 (+ 336us) server_negotiation.cc:299] Negotiation successful
0811 02:54:45.895622 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260811 02:54:45.895819 30199 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:54:45.891688 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.138.194:45119 (local address 127.12.138.193:46185)
0811 02:54:45.892058 (+ 370us) negotiation.cc:107] Waiting for socket to connect
0811 02:54:45.892069 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0811 02:54:45.892196 (+ 127us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:54:45.892503 (+ 307us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:54:45.892506 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:54:45.892522 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 02:54:45.892642 (+ 120us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:54:45.892647 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:54:45.893453 (+ 806us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:54:45.893456 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:54:45.894228 (+ 772us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:54:45.894236 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:54:45.895186 (+ 950us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:54:45.895189 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:54:45.895330 (+ 141us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:54:45.895435 (+ 105us) client_negotiation.cc:769] Sending connection context
0811 02:54:45.895521 (+ 86us) client_negotiation.cc:240] Negotiation successful
0811 02:54:45.895632 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":318,"mutex_wait_us":58,"thread_start_us":93,"threads_started":1}
I20260811 02:54:45.895826 30200 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:54:45.891586 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.138.195:46749 (local address 127.12.138.193:53999)
0811 02:54:45.891985 (+ 399us) negotiation.cc:107] Waiting for socket to connect
0811 02:54:45.891997 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 02:54:45.892160 (+ 163us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:54:45.892462 (+ 302us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:54:45.892465 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:54:45.892481 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 02:54:45.892620 (+ 139us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:54:45.892627 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:54:45.893389 (+ 762us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:54:45.893393 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:54:45.894309 (+ 916us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:54:45.894320 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:54:45.895174 (+ 854us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:54:45.895178 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:54:45.895349 (+ 171us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:54:45.895438 (+ 89us) client_negotiation.cc:769] Sending connection context
0811 02:54:45.895521 (+ 83us) client_negotiation.cc:240] Negotiation successful
0811 02:54:45.895632 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":328,"mutex_wait_us":102,"thread_start_us":118,"threads_started":1}
W20260811 02:54:45.896144 30025 tablet.cc:2369] T 89669c29a2d34fa98ad0180c930e32dd P 8a43edab73754e09a4b5e387d083f139: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 02:54:45.896512 29958 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "89669c29a2d34fa98ad0180c930e32dd" candidate_uuid: "13a904a6ad6a46a78dd222ee22eaea3c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a43edab73754e09a4b5e387d083f139" is_pre_election: true
I20260811 02:54:45.896698 30125 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "89669c29a2d34fa98ad0180c930e32dd" candidate_uuid: "13a904a6ad6a46a78dd222ee22eaea3c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "11f70dbb696346a491c8b80eaf7ce67f" is_pre_election: true
I20260811 02:54:45.896883 29958 raft_consensus.cc:2454] T 89669c29a2d34fa98ad0180c930e32dd P 8a43edab73754e09a4b5e387d083f139 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 13a904a6ad6a46a78dd222ee22eaea3c in term 0.
I20260811 02:54:45.897042 30125 raft_consensus.cc:2454] T 89669c29a2d34fa98ad0180c930e32dd P 11f70dbb696346a491c8b80eaf7ce67f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 13a904a6ad6a46a78dd222ee22eaea3c in term 0.
I20260811 02:54:45.897423 29733 leader_election.cc:302] T 89669c29a2d34fa98ad0180c930e32dd P 13a904a6ad6a46a78dd222ee22eaea3c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 13a904a6ad6a46a78dd222ee22eaea3c, 8a43edab73754e09a4b5e387d083f139; no voters:
I20260811 02:54:45.897732 29867 raft_consensus.cc:2781] T 89669c29a2d34fa98ad0180c930e32dd P 13a904a6ad6a46a78dd222ee22eaea3c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 02:54:45.897817 29867 raft_consensus.cc:482] T 89669c29a2d34fa98ad0180c930e32dd P 13a904a6ad6a46a78dd222ee22eaea3c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 02:54:45.897850 29867 raft_consensus.cc:3037] T 89669c29a2d34fa98ad0180c930e32dd P 13a904a6ad6a46a78dd222ee22eaea3c [term 0 FOLLOWER]: Advancing to term 1
I20260811 02:54:45.898931 29867 raft_consensus.cc:504] T 89669c29a2d34fa98ad0180c930e32dd P 13a904a6ad6a46a78dd222ee22eaea3c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "11f70dbb696346a491c8b80eaf7ce67f" member_type: VOTER last_known_addr { host: "127.12.138.195" port: 46749 } } peers { permanent_uuid: "8a43edab73754e09a4b5e387d083f139" member_type: VOTER last_known_addr { host: "127.12.138.194" port: 45119 } } peers { permanent_uuid: "13a904a6ad6a46a78dd222ee22eaea3c" member_type: VOTER last_known_addr { host: "127.12.138.193" port: 40085 } }
I20260811 02:54:45.899307 29867 leader_election.cc:288] T 89669c29a2d34fa98ad0180c930e32dd P 13a904a6ad6a46a78dd222ee22eaea3c [CANDIDATE]: Term 1 election: Requested vote from peers 11f70dbb696346a491c8b80eaf7ce67f (127.12.138.195:46749), 8a43edab73754e09a4b5e387d083f139 (127.12.138.194:45119)
I20260811 02:54:45.899751 30125 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "89669c29a2d34fa98ad0180c930e32dd" candidate_uuid: "13a904a6ad6a46a78dd222ee22eaea3c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "11f70dbb696346a491c8b80eaf7ce67f"
I20260811 02:54:45.899864 30125 raft_consensus.cc:3037] T 89669c29a2d34fa98ad0180c930e32dd P 11f70dbb696346a491c8b80eaf7ce67f [term 0 FOLLOWER]: Advancing to term 1
I20260811 02:54:45.901090 30125 raft_consensus.cc:2454] T 89669c29a2d34fa98ad0180c930e32dd P 11f70dbb696346a491c8b80eaf7ce67f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 13a904a6ad6a46a78dd222ee22eaea3c in term 1.
I20260811 02:54:45.901463 29734 leader_election.cc:302] T 89669c29a2d34fa98ad0180c930e32dd P 13a904a6ad6a46a78dd222ee22eaea3c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11f70dbb696346a491c8b80eaf7ce67f, 13a904a6ad6a46a78dd222ee22eaea3c; no voters:
I20260811 02:54:45.901429 29958 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "89669c29a2d34fa98ad0180c930e32dd" candidate_uuid: "13a904a6ad6a46a78dd222ee22eaea3c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a43edab73754e09a4b5e387d083f139"
I20260811 02:54:45.901578 29958 raft_consensus.cc:3037] T 89669c29a2d34fa98ad0180c930e32dd P 8a43edab73754e09a4b5e387d083f139 [term 0 FOLLOWER]: Advancing to term 1
I20260811 02:54:45.901697 29867 raft_consensus.cc:2781] T 89669c29a2d34fa98ad0180c930e32dd P 13a904a6ad6a46a78dd222ee22eaea3c [term 1 FOLLOWER]: Leader election won for term 1
I20260811 02:54:45.901916 29867 raft_consensus.cc:686] T 89669c29a2d34fa98ad0180c930e32dd P 13a904a6ad6a46a78dd222ee22eaea3c [term 1 LEADER]: Becoming Leader. State: Replica: 13a904a6ad6a46a78dd222ee22eaea3c, State: Running, Role: LEADER
I20260811 02:54:45.902176 29867 consensus_queue.cc:237] T 89669c29a2d34fa98ad0180c930e32dd P 13a904a6ad6a46a78dd222ee22eaea3c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "11f70dbb696346a491c8b80eaf7ce67f" member_type: VOTER last_known_addr { host: "127.12.138.195" port: 46749 } } peers { permanent_uuid: "8a43edab73754e09a4b5e387d083f139" member_type: VOTER last_known_addr { host: "127.12.138.194" port: 45119 } } peers { permanent_uuid: "13a904a6ad6a46a78dd222ee22eaea3c" member_type: VOTER last_known_addr { host: "127.12.138.193" port: 40085 } }
I20260811 02:54:45.902971 29958 raft_consensus.cc:2454] T 89669c29a2d34fa98ad0180c930e32dd P 8a43edab73754e09a4b5e387d083f139 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 13a904a6ad6a46a78dd222ee22eaea3c in term 1.
I20260811 02:54:45.905478 29642 catalog_manager.cc:5495] T 89669c29a2d34fa98ad0180c930e32dd P 13a904a6ad6a46a78dd222ee22eaea3c reported cstate change: term changed from 0 to 1, leader changed from <none> to 13a904a6ad6a46a78dd222ee22eaea3c (127.12.138.193). New cstate: current_term: 1 leader_uuid: "13a904a6ad6a46a78dd222ee22eaea3c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "11f70dbb696346a491c8b80eaf7ce67f" member_type: VOTER last_known_addr { host: "127.12.138.195" port: 46749 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8a43edab73754e09a4b5e387d083f139" member_type: VOTER last_known_addr { host: "127.12.138.194" port: 45119 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "13a904a6ad6a46a78dd222ee22eaea3c" member_type: VOTER last_known_addr { host: "127.12.138.193" port: 40085 } health_report { overall_health: HEALTHY } } }
W20260811 02:54:45.920734 29861 tablet.cc:2369] T 89669c29a2d34fa98ad0180c930e32dd P 13a904a6ad6a46a78dd222ee22eaea3c: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 02:54:45.972503 30205 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:54:45.968564 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45554 (local address 127.12.138.193:40085)
0811 02:54:45.968971 (+ 407us) server_negotiation.cc:206] Beginning negotiation
0811 02:54:45.968977 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 02:54:45.968991 (+ 14us) server_negotiation.cc:407] Connection header received
0811 02:54:45.969072 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:54:45.969074 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:54:45.969128 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:54:45.969301 (+ 173us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 02:54:45.969584 (+ 283us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:54:45.970118 (+ 534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:54:45.970959 (+ 841us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:54:45.971193 (+ 234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:54:45.971307 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:54:45.971605 (+ 298us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 02:54:45.972071 (+ 466us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 02:54:45.972202 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0811 02:54:45.972269 (+ 67us) server_negotiation.cc:299] Negotiation successful
0811 02:54:45.972328 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":326,"thread_start_us":117,"threads_started":1}
I20260811 02:54:45.976877 29958 raft_consensus.cc:1261] T 89669c29a2d34fa98ad0180c930e32dd P 8a43edab73754e09a4b5e387d083f139 [term 1 FOLLOWER]: Refusing update from remote peer 13a904a6ad6a46a78dd222ee22eaea3c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 02:54:45.976883 30125 raft_consensus.cc:1261] T 89669c29a2d34fa98ad0180c930e32dd P 11f70dbb696346a491c8b80eaf7ce67f [term 1 FOLLOWER]: Refusing update from remote peer 13a904a6ad6a46a78dd222ee22eaea3c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 02:54:45.977844 29867 consensus_queue.cc:1035] T 89669c29a2d34fa98ad0180c930e32dd P 13a904a6ad6a46a78dd222ee22eaea3c [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a43edab73754e09a4b5e387d083f139" member_type: VOTER last_known_addr { host: "127.12.138.194" port: 45119 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 02:54:45.978075 30201 consensus_queue.cc:1035] T 89669c29a2d34fa98ad0180c930e32dd P 13a904a6ad6a46a78dd222ee22eaea3c [LEADER]: Connected to new peer: Peer: permanent_uuid: "11f70dbb696346a491c8b80eaf7ce67f" member_type: VOTER last_known_addr { host: "127.12.138.195" port: 46749 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 02:54:45.986503 30208 mvcc.cc:205] Tried to move back new op lower bound from 7317163564953522176 to 7317163564658192384. Current Snapshot: MvccSnapshot[applied={T|T < 7317163564953522176 or (T in {7317163564953522176})}]
I20260811 02:54:45.987031 30209 mvcc.cc:205] Tried to move back new op lower bound from 7317163564953522176 to 7317163564658192384. Current Snapshot: MvccSnapshot[applied={T|T < 7317163564953522176}]
I20260811 02:54:45.995725 29644 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin'} at 127.0.0.1:58642:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 02:54:45.995975 29644 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin'} at 127.0.0.1:58642:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 02:54:45.998795 29644 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 11144008f58842b883c076177aaad052: Sending DeleteTablet for 3 replicas of tablet 89669c29a2d34fa98ad0180c930e32dd
I20260811 02:54:45.999747 29787 tablet_service.cc:1514] Processing DeleteTablet for tablet 89669c29a2d34fa98ad0180c930e32dd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 02:54:45 UTC) from {username='kudu', principal='kudu/127.12.138.254@KRBTEST.COM'} at 127.0.0.1:45552
I20260811 02:54:45.999797 30101 tablet_service.cc:1514] Processing DeleteTablet for tablet 89669c29a2d34fa98ad0180c930e32dd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 02:54:45 UTC) from {username='kudu', principal='kudu/127.12.138.254@KRBTEST.COM'} at 127.0.0.1:44732
I20260811 02:54:45.999854 29946 tablet_service.cc:1514] Processing DeleteTablet for tablet 89669c29a2d34fa98ad0180c930e32dd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 02:54:45 UTC) from {username='kudu', principal='kudu/127.12.138.254@KRBTEST.COM'} at 127.0.0.1:39906
I20260811 02:54:46.000331 12843 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDawEwS/build/debug/bin/kudu with pid 29689
I20260811 02:54:46.000559 30219 tablet_replica.cc:330] T 89669c29a2d34fa98ad0180c930e32dd P 8a43edab73754e09a4b5e387d083f139: stopping tablet replica
I20260811 02:54:46.004012 30219 raft_consensus.cc:2229] T 89669c29a2d34fa98ad0180c930e32dd P 8a43edab73754e09a4b5e387d083f139 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 02:54:46.004269 30219 raft_consensus.cc:2258] T 89669c29a2d34fa98ad0180c930e32dd P 8a43edab73754e09a4b5e387d083f139 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 02:54:46.000416 30220 tablet_replica.cc:330] T 89669c29a2d34fa98ad0180c930e32dd P 11f70dbb696346a491c8b80eaf7ce67f: stopping tablet replica
I20260811 02:54:46.006186 30220 raft_consensus.cc:2229] T 89669c29a2d34fa98ad0180c930e32dd P 11f70dbb696346a491c8b80eaf7ce67f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 02:54:46.006693 30220 raft_consensus.cc:2258] T 89669c29a2d34fa98ad0180c930e32dd P 11f70dbb696346a491c8b80eaf7ce67f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 02:54:46.008071 30219 ts_tablet_manager.cc:1905] T 89669c29a2d34fa98ad0180c930e32dd P 8a43edab73754e09a4b5e387d083f139: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 02:54:46.008968 30220 ts_tablet_manager.cc:1905] T 89669c29a2d34fa98ad0180c930e32dd P 11f70dbb696346a491c8b80eaf7ce67f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 02:54:46.010776 30219 ts_tablet_manager.cc:1918] T 89669c29a2d34fa98ad0180c930e32dd P 8a43edab73754e09a4b5e387d083f139: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 02:54:46.010897 30219 log.cc:1196] T 89669c29a2d34fa98ad0180c930e32dd P 8a43edab73754e09a4b5e387d083f139: Deleting WAL directory at /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786416707705494-12843-0/minicluster-data/ts-1/wal/wals/89669c29a2d34fa98ad0180c930e32dd
I20260811 02:54:46.011232 30219 ts_tablet_manager.cc:1939] T 89669c29a2d34fa98ad0180c930e32dd P 8a43edab73754e09a4b5e387d083f139: Deleting consensus metadata
I20260811 02:54:46.012298 29634 catalog_manager.cc:4841] TS 8a43edab73754e09a4b5e387d083f139 (127.12.138.194:45119): tablet 89669c29a2d34fa98ad0180c930e32dd (table test-table [id=b338f0c35cdd49408005b6579baff66b]) successfully deleted
I20260811 02:54:46.012633 30220 ts_tablet_manager.cc:1918] T 89669c29a2d34fa98ad0180c930e32dd P 11f70dbb696346a491c8b80eaf7ce67f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 02:54:46.012841 30220 log.cc:1196] T 89669c29a2d34fa98ad0180c930e32dd P 11f70dbb696346a491c8b80eaf7ce67f: Deleting WAL directory at /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786416707705494-12843-0/minicluster-data/ts-2/wal/wals/89669c29a2d34fa98ad0180c930e32dd
I20260811 02:54:46.013430 30220 ts_tablet_manager.cc:1939] T 89669c29a2d34fa98ad0180c930e32dd P 11f70dbb696346a491c8b80eaf7ce67f: Deleting consensus metadata
W20260811 02:54:46.014830 29634 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.12.138.193:40085 (error 108)
I20260811 02:54:46.014969 12843 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDawEwS/build/debug/bin/kudu with pid 29865
I20260811 02:54:46.014977 29635 catalog_manager.cc:4841] TS 11f70dbb696346a491c8b80eaf7ce67f (127.12.138.195:46749): tablet 89669c29a2d34fa98ad0180c930e32dd (table test-table [id=b338f0c35cdd49408005b6579baff66b]) successfully deleted
I20260811 02:54:46.019767 29869 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:54:46.019439 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.138.193:40085 (local address 127.0.0.1:45558)
0811 02:54:46.019587 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0811 02:54:46.019679 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.138.193:40085: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":60}
W20260811 02:54:46.019953 29634 catalog_manager.cc:4584] TS 13a904a6ad6a46a78dd222ee22eaea3c (127.12.138.193:40085): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 89669c29a2d34fa98ad0180c930e32dd: Network error: Client connection negotiation failed: client connection to 127.12.138.193:40085: connect: Connection refused (error 111)
I20260811 02:54:46.026060 12843 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDawEwS/build/debug/bin/kudu with pid 30027
I20260811 02:54:46.036669 12843 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDawEwS/build/debug/bin/kudu with pid 29615
I20260811 02:54:46.047982 12843 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDawEwS/build/debug/bin/kudu
/tmp/dist-test-taskDawEwS/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786416707705494-12843-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786416707705494-12843-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786416707705494-12843-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786416707705494-12843-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.138.254:45409
--webserver_interface=127.12.138.254
--webserver_port=46571
--keytab_file=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786416707705494-12843-0/krb5kdc/kudu_127.12.138.254.keytab
--principal=kudu/127.12.138.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.138.212:45835
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.12.138.254:45409
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786416707705494-12843-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786416707705494-12843-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786416707705494-12843-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 02:54:46.166829 30222 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 02:54:46.167161 30222 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 02:54:46.167239 30222 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 02:54:46.171339 30222 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 02:54:46.171446 30222 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 02:54:46.171487 30222 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 02:54:46.171528 30222 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 02:54:46.171568 30222 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 02:54:46.177379 30222 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.138.212:45835
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786416707705494-12843-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786416707705494-12843-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.12.138.254:45409
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786416707705494-12843-0/krb5kdc/kudu_127.12.138.254.keytab
--principal=kudu/127.12.138.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.138.254:45409
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786416707705494-12843-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.12.138.254
--webserver_port=46571
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30222
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786416707705494-12843-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 02:09:49 UTC on ccycloud.quasar-fsnqqc.root.comops.site
build id 4579570
I20260811 02:54:46.178572 30222 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 02:54:46.179625 30222 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 02:54:46.188701 30228 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:54:46.188786 30227 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:54:46.188791 30231 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:54:46.188985 30222 server_base.cc:1034] running on GCE node
I20260811 02:54:46.189623 30222 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 02:54:46.190783 30222 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 02:54:46.191998 30222 hybrid_clock.cc:648] HybridClock initialized: now 1786416886191984 us; error 26 us; skew 500 ppm
Aug 11 02:54:46 dist-test-slave-9gcw krb5kdc[28987](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416886, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.138.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 02:54:46.195113 30222 init.cc:377] Logged in from keytab as kudu/127.12.138.254@KRBTEST.COM (short username kudu)
I20260811 02:54:46.196237 30222 webserver.cc:460] Webserver started at http://127.12.138.254:46571/ using document root <none> and password file <none>
I20260811 02:54:46.196856 30222 fs_manager.cc:359] Metadata directory not provided
I20260811 02:54:46.196961 30222 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 02:54:46.201221 30222 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.000s
I20260811 02:54:46.203542 30239 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 02:54:46.204814 30222 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260811 02:54:46.204959 30222 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786416707705494-12843-0/minicluster-data/master-0/data,/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786416707705494-12843-0/minicluster-data/master-0/wal
uuid: "11144008f58842b883c076177aaad052"
format_stamp: "Formatted at 2026-08-11 02:54:43 on dist-test-slave-9gcw"
server_key: "2ee95749916bbaa0295a7c0d9ba9a695"
server_key_iv: "f7e08cdfd20e96649c8e5c4ac6e77e5d"
server_key_version: "encryptionkey@0"
I20260811 02:54:46.205593 30222 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786416707705494-12843-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786416707705494-12843-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786416707705494-12843-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 02:54:46.239316 30222 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 02:54:46.242352 30222 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 02:54:46.242573 30222 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 02:54:46.250190 30222 rpc_server.cc:307] RPC server started. Bound to: 127.12.138.254:45409
I20260811 02:54:46.250205 30309 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.138.254:45409 every 8 connection(s)
I20260811 02:54:46.251331 30222 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786416707705494-12843-0/minicluster-data/master-0/data/info.pb
I20260811 02:54:46.255728 30310 sys_catalog.cc:263] Verifying existing consensus state
I20260811 02:54:46.255805 12843 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDawEwS/build/debug/bin/kudu as pid 30222
I20260811 02:54:46.256228 12843 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDawEwS/build/debug/bin/kudu
/tmp/dist-test-taskDawEwS/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786416707705494-12843-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786416707705494-12843-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786416707705494-12843-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786416707705494-12843-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.138.193:40085
--local_ip_for_outbound_sockets=127.12.138.193
--tserver_master_addrs=127.12.138.254:45409
--webserver_port=34989
--webserver_interface=127.12.138.193
--keytab_file=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786416707705494-12843-0/krb5kdc/kudu_127.12.138.193.keytab
--principal=kudu/127.12.138.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.138.212:45835
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786416707705494-12843-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786416707705494-12843-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786416707705494-12843-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 02:54:46.257576 30310 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 11144008f58842b883c076177aaad052: Bootstrap starting.
I20260811 02:54:46.275161 30310 log.cc:824] T 00000000000000000000000000000000 P 11144008f58842b883c076177aaad052: Log is configured to *not* fsync() on all Append() calls
I20260811 02:54:46.286836 30310 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 11144008f58842b883c076177aaad052: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=7 ignored=0} mutations{seen=9 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 02:54:46.287266 30310 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 11144008f58842b883c076177aaad052: Bootstrap complete.
I20260811 02:54:46.291643 30310 raft_consensus.cc:348] T 00000000000000000000000000000000 P 11144008f58842b883c076177aaad052 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "11144008f58842b883c076177aaad052" member_type: VOTER last_known_addr { host: "127.12.138.254" port: 45409 } }
I20260811 02:54:46.292075 30310 raft_consensus.cc:729] T 00000000000000000000000000000000 P 11144008f58842b883c076177aaad052 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 11144008f58842b883c076177aaad052, State: Initialized, Role: FOLLOWER
I20260811 02:54:46.292555 30310 consensus_queue.cc:260] T 00000000000000000000000000000000 P 11144008f58842b883c076177aaad052 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "11144008f58842b883c076177aaad052" member_type: VOTER last_known_addr { host: "127.12.138.254" port: 45409 } }
I20260811 02:54:46.292712 30310 raft_consensus.cc:388] T 00000000000000000000000000000000 P 11144008f58842b883c076177aaad052 [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 02:54:46.292807 30310 raft_consensus.cc:482] T 00000000000000000000000000000000 P 11144008f58842b883c076177aaad052 [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 02:54:46.292920 30310 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 11144008f58842b883c076177aaad052 [term 2 FOLLOWER]: Advancing to term 3
I20260811 02:54:46.294986 30310 raft_consensus.cc:504] T 00000000000000000000000000000000 P 11144008f58842b883c076177aaad052 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "11144008f58842b883c076177aaad052" member_type: VOTER last_known_addr { host: "127.12.138.254" port: 45409 } }
I20260811 02:54:46.295377 30310 leader_election.cc:302] T 00000000000000000000000000000000 P 11144008f58842b883c076177aaad052 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 11144008f58842b883c076177aaad052; no voters:
I20260811 02:54:46.295735 30310 leader_election.cc:288] T 00000000000000000000000000000000 P 11144008f58842b883c076177aaad052 [CANDIDATE]: Term 3 election: Requested vote from peers
I20260811 02:54:46.295852 30314 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 11144008f58842b883c076177aaad052 [term 3 FOLLOWER]: Leader election won for term 3
I20260811 02:54:46.296051 30314 raft_consensus.cc:686] T 00000000000000000000000000000000 P 11144008f58842b883c076177aaad052 [term 3 LEADER]: Becoming Leader. State: Replica: 11144008f58842b883c076177aaad052, State: Running, Role: LEADER
I20260811 02:54:46.296356 30314 consensus_queue.cc:237] T 00000000000000000000000000000000 P 11144008f58842b883c076177aaad052 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "11144008f58842b883c076177aaad052" member_type: VOTER last_known_addr { host: "127.12.138.254" port: 45409 } }
I20260811 02:54:46.297015 30310 sys_catalog.cc:564] T 00000000000000000000000000000000 P 11144008f58842b883c076177aaad052 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 02:54:46.297333 30314 sys_catalog.cc:455] T 00000000000000000000000000000000 P 11144008f58842b883c076177aaad052 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 11144008f58842b883c076177aaad052. Latest consensus state: current_term: 3 leader_uuid: "11144008f58842b883c076177aaad052" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "11144008f58842b883c076177aaad052" member_type: VOTER last_known_addr { host: "127.12.138.254" port: 45409 } } }
I20260811 02:54:46.297461 30314 sys_catalog.cc:458] T 00000000000000000000000000000000 P 11144008f58842b883c076177aaad052 [sys.catalog]: This master's current role is: LEADER
I20260811 02:54:46.297855 30315 sys_catalog.cc:455] T 00000000000000000000000000000000 P 11144008f58842b883c076177aaad052 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "11144008f58842b883c076177aaad052" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "11144008f58842b883c076177aaad052" member_type: VOTER last_known_addr { host: "127.12.138.254" port: 45409 } } }
I20260811 02:54:46.298028 30315 sys_catalog.cc:458] T 00000000000000000000000000000000 P 11144008f58842b883c076177aaad052 [sys.catalog]: This master's current role is: LEADER
I20260811 02:54:46.299149 30318 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 02:54:46.304268 30318 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 02:54:46.304898 30318 catalog_manager.cc:1259] Loaded cluster ID: d19c1050b4cd4f2e8db89c9e58687fed
I20260811 02:54:46.304960 30318 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
F20260811 02:54:46.305950 30318 catalog_manager.cc:1466] Initializing Kudu internal certificate authority failed: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7f43de6c59ad google::LogMessage::Fail() at ??:0
@ 0x7f43de6c888a google::LogMessage::SendToLog() at ??:0
@ 0x7f43de6c54de google::LogMessage::Flush() at ??:0
@ 0x7f43de6c6a19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7f43e9004fdd _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlSt8functionIFNS_6StatusEvEERKNS_9consensus13RaftConsensusElPKcE_clES5_S9_lSB_ at ??:0
@ 0x7f43e9005d2f kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7f43e9002579 _ZZN4kudu6master14CatalogManager17ElectedAsLeaderCbEvENKUlvE_clEv at ??:0
@ 0x7f43e9046640 _ZNSt17_Function_handlerIFvvEZN4kudu6master14CatalogManager17ElectedAsLeaderCbEvEUlvE_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x61d838 std::function<>::operator()() at ??:0
@ 0x7f43e09173d7 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7f43e0917c83 _ZZN4kudu10ThreadPool12CreateThreadEvENKUlvE_clEv at ??:0
@ 0x7f43e09193f2 _ZNSt17_Function_handlerIFvvEZN4kudu10ThreadPool12CreateThreadEvEUlvE_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x61d838 std::function<>::operator()() at ??:0
@ 0x7f43e0908512 kudu::Thread::SuperviseThread() at ??:0
@ 0x7f43e1f4c6db start_thread at ??:0
@ 0x7f43dd1c361f clone at ??:0
W20260811 02:54:46.398483 30312 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 02:54:46.398753 30312 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 02:54:46.398805 30312 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 02:54:46.403065 30312 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 02:54:46.403127 30312 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 02:54:46.403239 30312 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.138.193
I20260811 02:54:46.408649 30312 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.138.212:45835
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786416707705494-12843-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786416707705494-12843-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786416707705494-12843-0/krb5kdc/kudu_127.12.138.193.keytab
--principal=kudu/127.12.138.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.138.193:40085
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786416707705494-12843-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.12.138.193
--webserver_port=34989
--tserver_master_addrs=127.12.138.254:45409
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30312
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.138.193
--log_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786416707705494-12843-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 02:09:49 UTC on ccycloud.quasar-fsnqqc.root.comops.site
build id 4579570
I20260811 02:54:46.409816 30312 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 02:54:46.410848 30312 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 02:54:46.420789 30357 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:54:46.420784 30358 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:54:46.420819 30361 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:54:46.421502 30312 server_base.cc:1034] running on GCE node
I20260811 02:54:46.421911 30312 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 02:54:46.422559 30312 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 02:54:46.423738 30312 hybrid_clock.cc:648] HybridClock initialized: now 1786416886423715 us; error 47 us; skew 500 ppm
Aug 11 02:54:46 dist-test-slave-9gcw krb5kdc[28987](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416886, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.138.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 02:54:46.426566 30312 init.cc:377] Logged in from keytab as kudu/127.12.138.193@KRBTEST.COM (short username kudu)
I20260811 02:54:46.427570 30312 webserver.cc:460] Webserver started at http://127.12.138.193:34989/ using document root <none> and password file <none>
I20260811 02:54:46.428158 30312 fs_manager.cc:359] Metadata directory not provided
I20260811 02:54:46.428222 30312 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 02:54:46.432221 30312 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.000s
I20260811 02:54:46.434504 30369 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 02:54:46.435832 30312 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260811 02:54:46.435930 30312 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786416707705494-12843-0/minicluster-data/ts-0/data,/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786416707705494-12843-0/minicluster-data/ts-0/wal
uuid: "13a904a6ad6a46a78dd222ee22eaea3c"
format_stamp: "Formatted at 2026-08-11 02:54:43 on dist-test-slave-9gcw"
server_key: "03fc6e3876c6257fb7637fbb9ccc8ff4"
server_key_iv: "68781f4cd4ccd5db115f71fa3f511505"
server_key_version: "encryptionkey@0"
I20260811 02:54:46.436499 30312 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786416707705494-12843-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786416707705494-12843-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786416707705494-12843-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 02:54:46.449401 30312 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 02:54:46.452917 30312 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 02:54:46.453135 30312 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 02:54:46.453886 30312 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 02:54:46.455278 30378 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260811 02:54:46.461123 30312 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260811 02:54:46.461277 30312 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.006s user 0.000s sys 0.001s
I20260811 02:54:46.461359 30312 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260811 02:54:46.463070 30312 ts_tablet_manager.cc:610] Registered 1 tablets
I20260811 02:54:46.463145 30312 ts_tablet_manager.cc:589] Time spent register tablets: real 0.002s user 0.000s sys 0.000s
I20260811 02:54:46.463496 30378 tablet_bootstrap.cc:492] T 89669c29a2d34fa98ad0180c930e32dd P 13a904a6ad6a46a78dd222ee22eaea3c: Bootstrap starting.
I20260811 02:54:46.474826 30312 rpc_server.cc:307] RPC server started. Bound to: 127.12.138.193:40085
I20260811 02:54:46.475083 30511 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.138.193:40085 every 8 connection(s)
I20260811 02:54:46.475965 30312 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786416707705494-12843-0/minicluster-data/ts-0/data/info.pb
I20260811 02:54:46.484833 12843 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDawEwS/build/debug/bin/kudu as pid 30312
I20260811 02:54:46.485383 12843 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDawEwS/build/debug/bin/kudu
/tmp/dist-test-taskDawEwS/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786416707705494-12843-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786416707705494-12843-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786416707705494-12843-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786416707705494-12843-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.138.194:45119
--local_ip_for_outbound_sockets=127.12.138.194
--tserver_master_addrs=127.12.138.254:45409
--webserver_port=38055
--webserver_interface=127.12.138.194
--keytab_file=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786416707705494-12843-0/krb5kdc/kudu_127.12.138.194.keytab
--principal=kudu/127.12.138.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.138.212:45835
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786416707705494-12843-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786416707705494-12843-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786416707705494-12843-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 02:54:46.486083 30378 log.cc:824] T 89669c29a2d34fa98ad0180c930e32dd P 13a904a6ad6a46a78dd222ee22eaea3c: Log is configured to *not* fsync() on all Append() calls
I20260811 02:54:46.493546 30378 tablet_bootstrap.cc:492] T 89669c29a2d34fa98ad0180c930e32dd P 13a904a6ad6a46a78dd222ee22eaea3c: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 02:54:46.493939 30378 tablet_bootstrap.cc:492] T 89669c29a2d34fa98ad0180c930e32dd P 13a904a6ad6a46a78dd222ee22eaea3c: Bootstrap complete.
I20260811 02:54:46.495035 30378 ts_tablet_manager.cc:1397] T 89669c29a2d34fa98ad0180c930e32dd P 13a904a6ad6a46a78dd222ee22eaea3c: Time spent bootstrapping tablet: real 0.032s user 0.025s sys 0.004s
I20260811 02:54:46.497921 30378 raft_consensus.cc:348] T 89669c29a2d34fa98ad0180c930e32dd P 13a904a6ad6a46a78dd222ee22eaea3c [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "11f70dbb696346a491c8b80eaf7ce67f" member_type: VOTER last_known_addr { host: "127.12.138.195" port: 46749 } } peers { permanent_uuid: "8a43edab73754e09a4b5e387d083f139" member_type: VOTER last_known_addr { host: "127.12.138.194" port: 45119 } } peers { permanent_uuid: "13a904a6ad6a46a78dd222ee22eaea3c" member_type: VOTER last_known_addr { host: "127.12.138.193" port: 40085 } }
I20260811 02:54:46.498447 30378 raft_consensus.cc:729] T 89669c29a2d34fa98ad0180c930e32dd P 13a904a6ad6a46a78dd222ee22eaea3c [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 13a904a6ad6a46a78dd222ee22eaea3c, State: Initialized, Role: FOLLOWER
I20260811 02:54:46.499024 30378 consensus_queue.cc:260] T 89669c29a2d34fa98ad0180c930e32dd P 13a904a6ad6a46a78dd222ee22eaea3c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "11f70dbb696346a491c8b80eaf7ce67f" member_type: VOTER last_known_addr { host: "127.12.138.195" port: 46749 } } peers { permanent_uuid: "8a43edab73754e09a4b5e387d083f139" member_type: VOTER last_known_addr { host: "127.12.138.194" port: 45119 } } peers { permanent_uuid: "13a904a6ad6a46a78dd222ee22eaea3c" member_type: VOTER last_known_addr { host: "127.12.138.193" port: 40085 } }
I20260811 02:54:46.500005 30378 ts_tablet_manager.cc:1428] T 89669c29a2d34fa98ad0180c930e32dd P 13a904a6ad6a46a78dd222ee22eaea3c: Time spent starting tablet: real 0.005s user 0.002s sys 0.004s
W20260811 02:54:46.531921 30514 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 02:54:46.478088 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.138.254:45409 (local address 127.12.138.193:51695)
0811 02:54:46.478584 (+ 496us) negotiation.cc:107] Waiting for socket to connect
0811 02:54:46.478638 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0811 02:54:46.479494 (+ 856us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:54:46.530849 (+ 51355us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.138.254:45409: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":285,"thread_start_us":149,"threads_started":1}
W20260811 02:54:46.532518 30512 heartbeater.cc:644] Failed to heartbeat to 127.12.138.254:45409 (0 consecutive failures): Network error: Failed to ping master at 127.12.138.254:45409: Client connection negotiation failed: client connection to 127.12.138.254:45409: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260811 02:54:46.533285 30514 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:54:46.533006 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.138.254:45409 (local address 127.12.138.193:35105)
0811 02:54:46.533135 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0811 02:54:46.533233 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.138.254:45409: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
I20260811 02:54:46.534027 30514 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:54:46.533823 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.138.254:45409 (local address 127.12.138.193:54655)
0811 02:54:46.533930 (+ 107us) negotiation.cc:107] Waiting for socket to connect
0811 02:54:46.533983 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.138.254:45409: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
W20260811 02:54:46.534225 30512 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260811 02:54:46.616156 30516 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 02:54:46.616428 30516 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 02:54:46.616473 30516 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 02:54:46.620579 30516 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 02:54:46.620652 30516 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 02:54:46.620754 30516 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.138.194
I20260811 02:54:46.626240 30516 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.138.212:45835
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786416707705494-12843-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786416707705494-12843-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786416707705494-12843-0/krb5kdc/kudu_127.12.138.194.keytab
--principal=kudu/127.12.138.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.138.194:45119
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786416707705494-12843-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.12.138.194
--webserver_port=38055
--tserver_master_addrs=127.12.138.254:45409
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30516
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.138.194
--log_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786416707705494-12843-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 02:09:49 UTC on ccycloud.quasar-fsnqqc.root.comops.site
build id 4579570
I20260811 02:54:46.627377 30516 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 02:54:46.628458 30516 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 02:54:46.638932 30526 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:54:46.638976 30524 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:54:46.639056 30516 server_base.cc:1034] running on GCE node
W20260811 02:54:46.638976 30522 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:54:46.639866 30516 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 02:54:46.640597 30516 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 02:54:46.641840 30516 hybrid_clock.cc:648] HybridClock initialized: now 1786416886641818 us; error 34 us; skew 500 ppm
Aug 11 02:54:46 dist-test-slave-9gcw krb5kdc[28987](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416886, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.138.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 02:54:46.645270 30516 init.cc:377] Logged in from keytab as kudu/127.12.138.194@KRBTEST.COM (short username kudu)
I20260811 02:54:46.646346 30516 webserver.cc:460] Webserver started at http://127.12.138.194:38055/ using document root <none> and password file <none>
I20260811 02:54:46.646996 30516 fs_manager.cc:359] Metadata directory not provided
I20260811 02:54:46.647073 30516 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 02:54:46.651505 30516 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.000s sys 0.004s
I20260811 02:54:46.653965 30533 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 02:54:46.655478 30516 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260811 02:54:46.655619 30516 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786416707705494-12843-0/minicluster-data/ts-1/data,/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786416707705494-12843-0/minicluster-data/ts-1/wal
uuid: "8a43edab73754e09a4b5e387d083f139"
format_stamp: "Formatted at 2026-08-11 02:54:44 on dist-test-slave-9gcw"
server_key: "0b24223102881f0eeb80a43fe932ab86"
server_key_iv: "1d7ad0a13c8dd59c653d43d0d1ab63ac"
server_key_version: "encryptionkey@0"
I20260811 02:54:46.656270 30516 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786416707705494-12843-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786416707705494-12843-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786416707705494-12843-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 02:54:46.675010 30516 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 02:54:46.678897 30516 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 02:54:46.679129 30516 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 02:54:46.679813 30516 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 02:54:46.680764 30516 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 02:54:46.680820 30516 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 02:54:46.680899 30516 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 02:54:46.680944 30516 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 02:54:46.691601 30516 rpc_server.cc:307] RPC server started. Bound to: 127.12.138.194:45119
I20260811 02:54:46.691622 30665 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.138.194:45119 every 8 connection(s)
I20260811 02:54:46.692761 30516 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786416707705494-12843-0/minicluster-data/ts-1/data/info.pb
I20260811 02:54:46.694732 12843 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDawEwS/build/debug/bin/kudu as pid 30516
I20260811 02:54:46.695191 12843 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDawEwS/build/debug/bin/kudu
/tmp/dist-test-taskDawEwS/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786416707705494-12843-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786416707705494-12843-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786416707705494-12843-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786416707705494-12843-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.138.195:46749
--local_ip_for_outbound_sockets=127.12.138.195
--tserver_master_addrs=127.12.138.254:45409
--webserver_port=42981
--webserver_interface=127.12.138.195
--keytab_file=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786416707705494-12843-0/krb5kdc/kudu_127.12.138.195.keytab
--principal=kudu/127.12.138.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.138.212:45835
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786416707705494-12843-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786416707705494-12843-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786416707705494-12843-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 02:54:46.697336 30668 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:54:46.695371 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.138.254:45409 (local address 127.12.138.194:41561)
0811 02:54:46.695933 (+ 562us) negotiation.cc:107] Waiting for socket to connect
0811 02:54:46.696120 (+ 187us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.138.254:45409: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":325,"thread_start_us":163,"threads_started":1}
W20260811 02:54:46.697882 30666 heartbeater.cc:644] Failed to heartbeat to 127.12.138.254:45409 (0 consecutive failures): Network error: Failed to ping master at 127.12.138.254:45409: Client connection negotiation failed: client connection to 127.12.138.254:45409: connect: Connection refused (error 111)
I20260811 02:54:46.698588 30668 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:54:46.698272 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.138.254:45409 (local address 127.12.138.194:56265)
0811 02:54:46.698427 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0811 02:54:46.698506 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.138.254:45409: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
I20260811 02:54:46.699342 30668 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:54:46.699032 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.138.254:45409 (local address 127.12.138.194:48655)
0811 02:54:46.699190 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0811 02:54:46.699257 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.138.254:45409: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
W20260811 02:54:46.699537 30666 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260811 02:54:46.813017 30669 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 02:54:46.813345 30669 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 02:54:46.813392 30669 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 02:54:46.817339 30669 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 02:54:46.817395 30669 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 02:54:46.817490 30669 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.138.195
I20260811 02:54:46.822659 30669 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.138.212:45835
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786416707705494-12843-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786416707705494-12843-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786416707705494-12843-0/krb5kdc/kudu_127.12.138.195.keytab
--principal=kudu/127.12.138.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.138.195:46749
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786416707705494-12843-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.12.138.195
--webserver_port=42981
--tserver_master_addrs=127.12.138.254:45409
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30669
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.138.195
--log_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786416707705494-12843-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 02:09:49 UTC on ccycloud.quasar-fsnqqc.root.comops.site
build id 4579570
I20260811 02:54:46.823808 30669 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 02:54:46.824851 30669 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 02:54:46.836097 30675 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:54:46.836175 30677 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:54:46.836098 30674 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:54:46.836465 30669 server_base.cc:1034] running on GCE node
I20260811 02:54:46.837006 30669 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 02:54:46.837787 30669 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 02:54:46.839013 30669 hybrid_clock.cc:648] HybridClock initialized: now 1786416886838981 us; error 56 us; skew 500 ppm
Aug 11 02:54:46 dist-test-slave-9gcw krb5kdc[28987](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416886, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.138.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 02:54:46.842317 30669 init.cc:377] Logged in from keytab as kudu/127.12.138.195@KRBTEST.COM (short username kudu)
I20260811 02:54:46.843256 30669 webserver.cc:460] Webserver started at http://127.12.138.195:42981/ using document root <none> and password file <none>
I20260811 02:54:46.843868 30669 fs_manager.cc:359] Metadata directory not provided
I20260811 02:54:46.843966 30669 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 02:54:46.848519 30669 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.000s
I20260811 02:54:46.850982 30686 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 02:54:46.852231 30669 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260811 02:54:46.852372 30669 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786416707705494-12843-0/minicluster-data/ts-2/data,/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786416707705494-12843-0/minicluster-data/ts-2/wal
uuid: "11f70dbb696346a491c8b80eaf7ce67f"
format_stamp: "Formatted at 2026-08-11 02:54:44 on dist-test-slave-9gcw"
server_key: "0eacae20ba2b9cf328c5a1f43f4d883a"
server_key_iv: "85600423a907dc8209ef1b09bcff9144"
server_key_version: "encryptionkey@0"
I20260811 02:54:46.853053 30669 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786416707705494-12843-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786416707705494-12843-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786416707705494-12843-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 02:54:46.873533 30669 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 02:54:46.876780 30669 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 02:54:46.876989 30669 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 02:54:46.877696 30669 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 02:54:46.878554 30669 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 02:54:46.878635 30669 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 02:54:46.878722 30669 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 02:54:46.878770 30669 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 02:54:46.888383 30669 rpc_server.cc:307] RPC server started. Bound to: 127.12.138.195:46749
I20260811 02:54:46.888407 30818 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.138.195:46749 every 8 connection(s)
I20260811 02:54:46.889500 30669 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786416707705494-12843-0/minicluster-data/ts-2/data/info.pb
I20260811 02:54:46.893800 30821 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:54:46.891940 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.138.254:45409 (local address 127.12.138.195:44021)
0811 02:54:46.892466 (+ 526us) negotiation.cc:107] Waiting for socket to connect
0811 02:54:46.892964 (+ 498us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.138.254:45409: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":320,"thread_start_us":124,"threads_started":1}
W20260811 02:54:46.894412 30819 heartbeater.cc:644] Failed to heartbeat to 127.12.138.254:45409 (0 consecutive failures): Network error: Failed to ping master at 127.12.138.254:45409: Client connection negotiation failed: client connection to 127.12.138.254:45409: connect: Connection refused (error 111)
I20260811 02:54:46.894503 12843 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDawEwS/build/debug/bin/kudu as pid 30669
I20260811 02:54:46.895098 30821 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:54:46.894872 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.138.254:45409 (local address 127.12.138.195:50669)
0811 02:54:46.894994 (+ 122us) negotiation.cc:107] Waiting for socket to connect
0811 02:54:46.895043 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.138.254:45409: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
I20260811 02:54:46.895818 30821 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:54:46.895619 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.138.254:45409 (local address 127.12.138.195:49963)
0811 02:54:46.895709 (+ 90us) negotiation.cc:107] Waiting for socket to connect
0811 02:54:46.895760 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.138.254:45409: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":26}
W20260811 02:54:46.896016 30819 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260811 02:54:47.535348 30823 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:54:47.534860 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.138.254:45409 (local address 127.12.138.193:59945)
0811 02:54:47.535157 (+ 297us) negotiation.cc:107] Waiting for socket to connect
0811 02:54:47.535219 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.138.254:45409: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":211,"thread_start_us":84,"threads_started":1}
I20260811 02:54:47.700641 30824 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:54:47.700176 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.138.254:45409 (local address 127.12.138.194:34297)
0811 02:54:47.700439 (+ 263us) negotiation.cc:107] Waiting for socket to connect
0811 02:54:47.700521 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.138.254:45409: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":181,"thread_start_us":75,"threads_started":1}
I20260811 02:54:47.897063 30825 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:54:47.896610 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.138.254:45409 (local address 127.12.138.195:33541)
0811 02:54:47.896891 (+ 281us) negotiation.cc:107] Waiting for socket to connect
0811 02:54:47.896953 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.138.254:45409: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":204,"thread_start_us":89,"threads_started":1}
I20260811 02:54:48.348946 30827 raft_consensus.cc:482] T 89669c29a2d34fa98ad0180c930e32dd P 13a904a6ad6a46a78dd222ee22eaea3c [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 02:54:48.349259 30827 raft_consensus.cc:504] T 89669c29a2d34fa98ad0180c930e32dd P 13a904a6ad6a46a78dd222ee22eaea3c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "11f70dbb696346a491c8b80eaf7ce67f" member_type: VOTER last_known_addr { host: "127.12.138.195" port: 46749 } } peers { permanent_uuid: "8a43edab73754e09a4b5e387d083f139" member_type: VOTER last_known_addr { host: "127.12.138.194" port: 45119 } } peers { permanent_uuid: "13a904a6ad6a46a78dd222ee22eaea3c" member_type: VOTER last_known_addr { host: "127.12.138.193" port: 40085 } }
I20260811 02:54:48.350621 30827 leader_election.cc:288] T 89669c29a2d34fa98ad0180c930e32dd P 13a904a6ad6a46a78dd222ee22eaea3c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 11f70dbb696346a491c8b80eaf7ce67f (127.12.138.195:46749), 8a43edab73754e09a4b5e387d083f139 (127.12.138.194:45119)
Aug 11 02:54:48 dist-test-slave-9gcw krb5kdc[28987](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416886, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.138.193@KRBTEST.COM for kudu/127.12.138.195@KRBTEST.COM
Aug 11 02:54:48 dist-test-slave-9gcw krb5kdc[28987](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416886, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.138.193@KRBTEST.COM for kudu/127.12.138.194@KRBTEST.COM
I20260811 02:54:48.365828 30828 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:54:48.350879 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.138.194:45119 (local address 127.12.138.193:34717)
0811 02:54:48.351359 (+ 480us) negotiation.cc:107] Waiting for socket to connect
0811 02:54:48.351376 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 02:54:48.351454 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:54:48.353270 (+ 1816us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:54:48.353273 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:54:48.353783 (+ 510us) client_negotiation.cc:189] Negotiated authn=SASL
0811 02:54:48.354273 (+ 490us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:54:48.354291 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:54:48.355627 (+ 1336us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:54:48.355632 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:54:48.356772 (+ 1140us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:54:48.356784 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:54:48.357680 (+ 896us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:54:48.357684 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:54:48.357780 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:54:48.358683 (+ 903us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 02:54:48.358696 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0811 02:54:48.360989 (+ 2293us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 02:54:48.363577 (+ 2588us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:54:48.363581 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:54:48.363583 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:54:48.363809 (+ 226us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:54:48.364192 (+ 383us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:54:48.364195 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:54:48.364197 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:54:48.364286 (+ 89us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:54:48.364934 (+ 648us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 02:54:48.364938 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 02:54:48.365049 (+ 111us) client_negotiation.cc:769] Sending connection context
0811 02:54:48.365248 (+ 199us) client_negotiation.cc:240] Negotiation successful
0811 02:54:48.365605 (+ 357us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":415,"mutex_wait_us":158,"spinlock_wait_cycles":19712,"thread_start_us":76,"threads_started":1}
I20260811 02:54:48.365883 30831 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:54:48.351159 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.138.193:34717 (local address 127.12.138.194:45119)
0811 02:54:48.351603 (+ 444us) server_negotiation.cc:206] Beginning negotiation
0811 02:54:48.351614 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 02:54:48.351649 (+ 35us) server_negotiation.cc:407] Connection header received
0811 02:54:48.352473 (+ 824us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:54:48.352481 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:54:48.352840 (+ 359us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:54:48.353181 (+ 341us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:54:48.354496 (+ 1315us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:54:48.355472 (+ 976us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:54:48.356909 (+ 1437us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:54:48.357539 (+ 630us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:54:48.357735 (+ 196us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:54:48.361228 (+ 3493us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:54:48.361234 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:54:48.361252 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:54:48.361286 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:54:48.363435 (+ 2149us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:54:48.363935 (+ 500us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:54:48.363941 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:54:48.363947 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:54:48.364078 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:54:48.364388 (+ 310us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:54:48.364393 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:54:48.364395 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:54:48.364815 (+ 420us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:54:48.364966 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0811 02:54:48.365373 (+ 407us) server_negotiation.cc:299] Negotiation successful
0811 02:54:48.365687 (+ 314us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":327,"thread_start_us":102,"threads_started":1}
I20260811 02:54:48.365878 30830 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:54:48.350996 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.138.193:59441 (local address 127.12.138.195:46749)
0811 02:54:48.351326 (+ 330us) server_negotiation.cc:206] Beginning negotiation
0811 02:54:48.351354 (+ 28us) server_negotiation.cc:399] Waiting for connection header
0811 02:54:48.351693 (+ 339us) server_negotiation.cc:407] Connection header received
0811 02:54:48.352502 (+ 809us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:54:48.352510 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:54:48.352837 (+ 327us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:54:48.353185 (+ 348us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:54:48.354463 (+ 1278us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:54:48.355467 (+ 1004us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:54:48.356782 (+ 1315us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:54:48.357459 (+ 677us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:54:48.357678 (+ 219us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:54:48.360956 (+ 3278us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:54:48.360961 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:54:48.360973 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:54:48.361002 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:54:48.363148 (+ 2146us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:54:48.363734 (+ 586us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:54:48.363740 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:54:48.363745 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:54:48.363878 (+ 133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:54:48.364218 (+ 340us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:54:48.364222 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:54:48.364224 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:54:48.364658 (+ 434us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:54:48.364817 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0811 02:54:48.365390 (+ 573us) server_negotiation.cc:299] Negotiation successful
0811 02:54:48.365674 (+ 284us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":183,"thread_start_us":62,"threads_started":1}
I20260811 02:54:48.365890 30829 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:54:48.350841 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.138.195:46749 (local address 127.12.138.193:59441)
0811 02:54:48.351163 (+ 322us) negotiation.cc:107] Waiting for socket to connect
0811 02:54:48.351179 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 02:54:48.351322 (+ 143us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:54:48.353234 (+ 1912us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:54:48.353240 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:54:48.353849 (+ 609us) client_negotiation.cc:189] Negotiated authn=SASL
0811 02:54:48.354274 (+ 425us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:54:48.354290 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:54:48.355638 (+ 1348us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:54:48.355642 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:54:48.356649 (+ 1007us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:54:48.356659 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:54:48.357585 (+ 926us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:54:48.357589 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:54:48.357773 (+ 184us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:54:48.358596 (+ 823us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 02:54:48.358622 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0811 02:54:48.360685 (+ 2063us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 02:54:48.363324 (+ 2639us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:54:48.363329 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:54:48.363347 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:54:48.363601 (+ 254us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:54:48.363980 (+ 379us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:54:48.363983 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:54:48.363985 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:54:48.364125 (+ 140us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:54:48.364779 (+ 654us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 02:54:48.364785 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 02:54:48.365014 (+ 229us) client_negotiation.cc:769] Sending connection context
0811 02:54:48.365256 (+ 242us) client_negotiation.cc:240] Negotiation successful
0811 02:54:48.365712 (+ 456us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":251,"mutex_wait_us":261,"spinlock_wait_cycles":2816,"thread_start_us":75,"threads_started":1}
W20260811 02:54:48.369207 30373 leader_election.cc:341] T 89669c29a2d34fa98ad0180c930e32dd P 13a904a6ad6a46a78dd222ee22eaea3c [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 8a43edab73754e09a4b5e387d083f139 (127.12.138.194:45119): Not found: Tablet not found: 89669c29a2d34fa98ad0180c930e32dd
W20260811 02:54:48.369696 30374 leader_election.cc:341] T 89669c29a2d34fa98ad0180c930e32dd P 13a904a6ad6a46a78dd222ee22eaea3c [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 11f70dbb696346a491c8b80eaf7ce67f (127.12.138.195:46749): Not found: Tablet not found: 89669c29a2d34fa98ad0180c930e32dd
I20260811 02:54:48.369803 30374 leader_election.cc:302] T 89669c29a2d34fa98ad0180c930e32dd P 13a904a6ad6a46a78dd222ee22eaea3c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 13a904a6ad6a46a78dd222ee22eaea3c; no voters: 11f70dbb696346a491c8b80eaf7ce67f, 8a43edab73754e09a4b5e387d083f139
I20260811 02:54:48.370139 30827 raft_consensus.cc:2726] T 89669c29a2d34fa98ad0180c930e32dd P 13a904a6ad6a46a78dd222ee22eaea3c [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260811 02:54:48.536533 30829 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:54:48.536221 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.138.254:45409 (local address 127.12.138.193:54539)
0811 02:54:48.536376 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0811 02:54:48.536447 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.138.254:45409: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":69}
I20260811 02:54:48.702136 30832 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:54:48.701453 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.138.254:45409 (local address 127.12.138.194:60573)
0811 02:54:48.701928 (+ 475us) negotiation.cc:107] Waiting for socket to connect
0811 02:54:48.701985 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.138.254:45409: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":399,"thread_start_us":93,"threads_started":1}
I20260811 02:54:48.898516 30833 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:54:48.898057 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.138.254:45409 (local address 127.12.138.195:35835)
0811 02:54:48.898352 (+ 295us) negotiation.cc:107] Waiting for socket to connect
0811 02:54:48.898418 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.138.254:45409: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":215,"thread_start_us":97,"threads_started":1}
I20260811 02:54:49.537966 30834 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:54:49.537419 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.138.254:45409 (local address 127.12.138.193:44035)
0811 02:54:49.537798 (+ 379us) negotiation.cc:107] Waiting for socket to connect
0811 02:54:49.537864 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.138.254:45409: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":294,"thread_start_us":84,"threads_started":1}
I20260811 02:54:49.703454 30836 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:54:49.703020 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.138.254:45409 (local address 127.12.138.194:55599)
0811 02:54:49.703289 (+ 269us) negotiation.cc:107] Waiting for socket to connect
0811 02:54:49.703353 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.138.254:45409: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":188,"thread_start_us":79,"threads_started":1}
I20260811 02:54:49.899928 30837 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:54:49.899342 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.138.254:45409 (local address 127.12.138.195:54547)
0811 02:54:49.899734 (+ 392us) negotiation.cc:107] Waiting for socket to connect
0811 02:54:49.899805 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.138.254:45409: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":235,"thread_start_us":101,"threads_started":1}
I20260811 02:54:50.424549 30838 raft_consensus.cc:482] T 89669c29a2d34fa98ad0180c930e32dd P 13a904a6ad6a46a78dd222ee22eaea3c [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 02:54:50.424676 30838 raft_consensus.cc:504] T 89669c29a2d34fa98ad0180c930e32dd P 13a904a6ad6a46a78dd222ee22eaea3c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "11f70dbb696346a491c8b80eaf7ce67f" member_type: VOTER last_known_addr { host: "127.12.138.195" port: 46749 } } peers { permanent_uuid: "8a43edab73754e09a4b5e387d083f139" member_type: VOTER last_known_addr { host: "127.12.138.194" port: 45119 } } peers { permanent_uuid: "13a904a6ad6a46a78dd222ee22eaea3c" member_type: VOTER last_known_addr { host: "127.12.138.193" port: 40085 } }
I20260811 02:54:50.425055 30838 leader_election.cc:288] T 89669c29a2d34fa98ad0180c930e32dd P 13a904a6ad6a46a78dd222ee22eaea3c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 11f70dbb696346a491c8b80eaf7ce67f (127.12.138.195:46749), 8a43edab73754e09a4b5e387d083f139 (127.12.138.194:45119)
W20260811 02:54:50.425956 30374 leader_election.cc:341] T 89669c29a2d34fa98ad0180c930e32dd P 13a904a6ad6a46a78dd222ee22eaea3c [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 11f70dbb696346a491c8b80eaf7ce67f (127.12.138.195:46749): Not found: Tablet not found: 89669c29a2d34fa98ad0180c930e32dd
W20260811 02:54:50.426054 30373 leader_election.cc:341] T 89669c29a2d34fa98ad0180c930e32dd P 13a904a6ad6a46a78dd222ee22eaea3c [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 8a43edab73754e09a4b5e387d083f139 (127.12.138.194:45119): Not found: Tablet not found: 89669c29a2d34fa98ad0180c930e32dd
I20260811 02:54:50.426164 30373 leader_election.cc:302] T 89669c29a2d34fa98ad0180c930e32dd P 13a904a6ad6a46a78dd222ee22eaea3c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 13a904a6ad6a46a78dd222ee22eaea3c; no voters: 11f70dbb696346a491c8b80eaf7ce67f, 8a43edab73754e09a4b5e387d083f139
I20260811 02:54:50.426330 30838 raft_consensus.cc:2726] T 89669c29a2d34fa98ad0180c930e32dd P 13a904a6ad6a46a78dd222ee22eaea3c [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260811 02:54:50.539186 30839 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:54:50.538718 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.138.254:45409 (local address 127.12.138.193:35717)
0811 02:54:50.539009 (+ 291us) negotiation.cc:107] Waiting for socket to connect
0811 02:54:50.539080 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.138.254:45409: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":215,"thread_start_us":97,"threads_started":1}
I20260811 02:54:50.704865 30840 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:54:50.704364 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.138.254:45409 (local address 127.12.138.194:50861)
0811 02:54:50.704664 (+ 300us) negotiation.cc:107] Waiting for socket to connect
0811 02:54:50.704739 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.138.254:45409: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":204,"thread_start_us":93,"threads_started":1}
I20260811 02:54:50.901415 30841 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:54:50.900860 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.138.254:45409 (local address 127.12.138.195:48985)
0811 02:54:50.901244 (+ 384us) negotiation.cc:107] Waiting for socket to connect
0811 02:54:50.901311 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.138.254:45409: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":222,"thread_start_us":102,"threads_started":1}
I20260811 02:54:51.540669 30842 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:54:51.540198 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.138.254:45409 (local address 127.12.138.193:51323)
0811 02:54:51.540508 (+ 310us) negotiation.cc:107] Waiting for socket to connect
0811 02:54:51.540572 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.138.254:45409: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":232,"thread_start_us":84,"threads_started":1}
I20260811 02:54:51.706164 30843 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:54:51.705725 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.138.254:45409 (local address 127.12.138.194:49741)
0811 02:54:51.705998 (+ 273us) negotiation.cc:107] Waiting for socket to connect
0811 02:54:51.706066 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.138.254:45409: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":194,"thread_start_us":90,"threads_started":1}
I20260811 02:54:51.902948 30844 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:54:51.902464 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.138.254:45409 (local address 127.12.138.195:45897)
0811 02:54:51.902779 (+ 315us) negotiation.cc:107] Waiting for socket to connect
0811 02:54:51.902844 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.138.254:45409: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":195,"thread_start_us":81,"threads_started":1}
I20260811 02:54:52.542027 30846 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:54:52.541550 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.138.254:45409 (local address 127.12.138.193:57815)
0811 02:54:52.541855 (+ 305us) negotiation.cc:107] Waiting for socket to connect
0811 02:54:52.541924 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.138.254:45409: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":221,"thread_start_us":102,"threads_started":1}
I20260811 02:54:52.707450 30847 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:54:52.707025 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.138.254:45409 (local address 127.12.138.194:44841)
0811 02:54:52.707291 (+ 266us) negotiation.cc:107] Waiting for socket to connect
0811 02:54:52.707351 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.138.254:45409: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":190,"thread_start_us":90,"threads_started":1}
I20260811 02:54:52.904170 30848 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:54:52.903770 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.138.254:45409 (local address 127.12.138.195:39035)
0811 02:54:52.904020 (+ 250us) negotiation.cc:107] Waiting for socket to connect
0811 02:54:52.904076 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.138.254:45409: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":181,"thread_start_us":84,"threads_started":1}
I20260811 02:54:53.513088 30849 raft_consensus.cc:482] T 89669c29a2d34fa98ad0180c930e32dd P 13a904a6ad6a46a78dd222ee22eaea3c [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 02:54:53.513322 30849 raft_consensus.cc:504] T 89669c29a2d34fa98ad0180c930e32dd P 13a904a6ad6a46a78dd222ee22eaea3c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "11f70dbb696346a491c8b80eaf7ce67f" member_type: VOTER last_known_addr { host: "127.12.138.195" port: 46749 } } peers { permanent_uuid: "8a43edab73754e09a4b5e387d083f139" member_type: VOTER last_known_addr { host: "127.12.138.194" port: 45119 } } peers { permanent_uuid: "13a904a6ad6a46a78dd222ee22eaea3c" member_type: VOTER last_known_addr { host: "127.12.138.193" port: 40085 } }
I20260811 02:54:53.513936 30849 leader_election.cc:288] T 89669c29a2d34fa98ad0180c930e32dd P 13a904a6ad6a46a78dd222ee22eaea3c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 11f70dbb696346a491c8b80eaf7ce67f (127.12.138.195:46749), 8a43edab73754e09a4b5e387d083f139 (127.12.138.194:45119)
W20260811 02:54:53.514730 30373 leader_election.cc:341] T 89669c29a2d34fa98ad0180c930e32dd P 13a904a6ad6a46a78dd222ee22eaea3c [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 8a43edab73754e09a4b5e387d083f139 (127.12.138.194:45119): Not found: Tablet not found: 89669c29a2d34fa98ad0180c930e32dd
W20260811 02:54:53.514886 30374 leader_election.cc:341] T 89669c29a2d34fa98ad0180c930e32dd P 13a904a6ad6a46a78dd222ee22eaea3c [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 11f70dbb696346a491c8b80eaf7ce67f (127.12.138.195:46749): Not found: Tablet not found: 89669c29a2d34fa98ad0180c930e32dd
I20260811 02:54:53.514982 30374 leader_election.cc:302] T 89669c29a2d34fa98ad0180c930e32dd P 13a904a6ad6a46a78dd222ee22eaea3c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 13a904a6ad6a46a78dd222ee22eaea3c; no voters: 11f70dbb696346a491c8b80eaf7ce67f, 8a43edab73754e09a4b5e387d083f139
I20260811 02:54:53.515177 30849 raft_consensus.cc:2726] T 89669c29a2d34fa98ad0180c930e32dd P 13a904a6ad6a46a78dd222ee22eaea3c [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260811 02:54:53.543177 30850 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:54:53.542761 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.138.254:45409 (local address 127.12.138.193:52753)
0811 02:54:53.543026 (+ 265us) negotiation.cc:107] Waiting for socket to connect
0811 02:54:53.543082 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.138.254:45409: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":189,"thread_start_us":92,"threads_started":1}
I20260811 02:54:53.708837 30852 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:54:53.708329 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.138.254:45409 (local address 127.12.138.194:38493)
0811 02:54:53.708673 (+ 344us) negotiation.cc:107] Waiting for socket to connect
0811 02:54:53.708731 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.138.254:45409: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":261,"thread_start_us":100,"threads_started":1}
I20260811 02:54:53.905589 30853 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:54:53.904986 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.138.254:45409 (local address 127.12.138.195:36773)
0811 02:54:53.905433 (+ 447us) negotiation.cc:107] Waiting for socket to connect
0811 02:54:53.905491 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.138.254:45409: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":374,"thread_start_us":207,"threads_started":1}
I20260811 02:54:54.544370 30854 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:54:54.543922 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.138.254:45409 (local address 127.12.138.193:57293)
0811 02:54:54.544200 (+ 278us) negotiation.cc:107] Waiting for socket to connect
0811 02:54:54.544265 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.138.254:45409: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":191,"thread_start_us":80,"threads_started":1}
I20260811 02:54:54.710036 30855 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:54:54.709607 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.138.254:45409 (local address 127.12.138.194:59711)
0811 02:54:54.709877 (+ 270us) negotiation.cc:107] Waiting for socket to connect
0811 02:54:54.709932 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.138.254:45409: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":196,"thread_start_us":93,"threads_started":1}
I20260811 02:54:54.907013 30856 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:54:54.906426 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.138.254:45409 (local address 127.12.138.195:43911)
0811 02:54:54.906760 (+ 334us) negotiation.cc:107] Waiting for socket to connect
0811 02:54:54.906864 (+ 104us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.138.254:45409: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":241,"thread_start_us":153,"threads_started":1}
I20260811 02:54:55.545835 30857 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:54:55.545357 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.138.254:45409 (local address 127.12.138.193:53801)
0811 02:54:55.545650 (+ 293us) negotiation.cc:107] Waiting for socket to connect
0811 02:54:55.545728 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.138.254:45409: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":207,"thread_start_us":93,"threads_started":1}
I20260811 02:54:55.711470 30859 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:54:55.710989 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.138.254:45409 (local address 127.12.138.194:41653)
0811 02:54:55.711303 (+ 314us) negotiation.cc:107] Waiting for socket to connect
0811 02:54:55.711366 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.138.254:45409: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":231,"thread_start_us":96,"threads_started":1}
I20260811 02:54:55.830067 30861 raft_consensus.cc:482] T 89669c29a2d34fa98ad0180c930e32dd P 13a904a6ad6a46a78dd222ee22eaea3c [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 02:54:55.830246 30861 raft_consensus.cc:504] T 89669c29a2d34fa98ad0180c930e32dd P 13a904a6ad6a46a78dd222ee22eaea3c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "11f70dbb696346a491c8b80eaf7ce67f" member_type: VOTER last_known_addr { host: "127.12.138.195" port: 46749 } } peers { permanent_uuid: "8a43edab73754e09a4b5e387d083f139" member_type: VOTER last_known_addr { host: "127.12.138.194" port: 45119 } } peers { permanent_uuid: "13a904a6ad6a46a78dd222ee22eaea3c" member_type: VOTER last_known_addr { host: "127.12.138.193" port: 40085 } }
I20260811 02:54:55.830740 30861 leader_election.cc:288] T 89669c29a2d34fa98ad0180c930e32dd P 13a904a6ad6a46a78dd222ee22eaea3c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 11f70dbb696346a491c8b80eaf7ce67f (127.12.138.195:46749), 8a43edab73754e09a4b5e387d083f139 (127.12.138.194:45119)
W20260811 02:54:55.831597 30374 leader_election.cc:341] T 89669c29a2d34fa98ad0180c930e32dd P 13a904a6ad6a46a78dd222ee22eaea3c [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 11f70dbb696346a491c8b80eaf7ce67f (127.12.138.195:46749): Not found: Tablet not found: 89669c29a2d34fa98ad0180c930e32dd
W20260811 02:54:55.831720 30373 leader_election.cc:341] T 89669c29a2d34fa98ad0180c930e32dd P 13a904a6ad6a46a78dd222ee22eaea3c [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 8a43edab73754e09a4b5e387d083f139 (127.12.138.194:45119): Not found: Tablet not found: 89669c29a2d34fa98ad0180c930e32dd
I20260811 02:54:55.831802 30373 leader_election.cc:302] T 89669c29a2d34fa98ad0180c930e32dd P 13a904a6ad6a46a78dd222ee22eaea3c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 13a904a6ad6a46a78dd222ee22eaea3c; no voters: 11f70dbb696346a491c8b80eaf7ce67f, 8a43edab73754e09a4b5e387d083f139
I20260811 02:54:55.831947 30861 raft_consensus.cc:2726] T 89669c29a2d34fa98ad0180c930e32dd P 13a904a6ad6a46a78dd222ee22eaea3c [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260811 02:54:55.908526 30862 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:54:55.907977 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.138.254:45409 (local address 127.12.138.195:57049)
0811 02:54:55.908340 (+ 363us) negotiation.cc:107] Waiting for socket to connect
0811 02:54:55.908410 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.138.254:45409: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":259,"thread_start_us":102,"threads_started":1}
I20260811 02:54:56.547237 30863 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:54:56.546781 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.138.254:45409 (local address 127.12.138.193:54797)
0811 02:54:56.547057 (+ 276us) negotiation.cc:107] Waiting for socket to connect
0811 02:54:56.547126 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.138.254:45409: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":197,"thread_start_us":78,"threads_started":1}
I20260811 02:54:56.713313 30864 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:54:56.712507 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.138.254:45409 (local address 127.12.138.194:54233)
0811 02:54:56.712995 (+ 488us) negotiation.cc:107] Waiting for socket to connect
0811 02:54:56.713089 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.138.254:45409: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":363,"thread_start_us":215,"threads_started":1}
I20260811 02:54:56.909919 30865 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:54:56.909392 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.138.254:45409 (local address 127.12.138.195:60333)
0811 02:54:56.909742 (+ 350us) negotiation.cc:107] Waiting for socket to connect
0811 02:54:56.909811 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.138.254:45409: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":262,"thread_start_us":95,"threads_started":1}
I20260811 02:54:57.548454 30866 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:54:57.548043 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.138.254:45409 (local address 127.12.138.193:51627)
0811 02:54:57.548293 (+ 250us) negotiation.cc:107] Waiting for socket to connect
0811 02:54:57.548353 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.138.254:45409: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":178,"thread_start_us":76,"threads_started":1}
I20260811 02:54:57.714583 30867 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:54:57.714130 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.138.254:45409 (local address 127.12.138.194:45211)
0811 02:54:57.714417 (+ 287us) negotiation.cc:107] Waiting for socket to connect
0811 02:54:57.714477 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.138.254:45409: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":202,"thread_start_us":95,"threads_started":1}
I20260811 02:54:57.911383 30868 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:54:57.910872 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.138.254:45409 (local address 127.12.138.195:60327)
0811 02:54:57.911217 (+ 345us) negotiation.cc:107] Waiting for socket to connect
0811 02:54:57.911283 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.138.254:45409: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":240,"thread_start_us":102,"threads_started":1}
I20260811 02:54:58.549856 30869 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:54:58.549401 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.138.254:45409 (local address 127.12.138.193:60347)
0811 02:54:58.549689 (+ 288us) negotiation.cc:107] Waiting for socket to connect
0811 02:54:58.549755 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.138.254:45409: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":208,"thread_start_us":82,"threads_started":1}
I20260811 02:54:58.715963 30870 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:54:58.715437 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.138.254:45409 (local address 127.12.138.194:49243)
0811 02:54:58.715760 (+ 323us) negotiation.cc:107] Waiting for socket to connect
0811 02:54:58.715834 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.138.254:45409: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":230,"thread_start_us":96,"threads_started":1}
I20260811 02:54:58.912688 30871 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:54:58.912247 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.138.254:45409 (local address 127.12.138.195:50821)
0811 02:54:58.912521 (+ 274us) negotiation.cc:107] Waiting for socket to connect
0811 02:54:58.912583 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.138.254:45409: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":199,"thread_start_us":79,"threads_started":1}
I20260811 02:54:59.551314 30872 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:54:59.550837 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.138.254:45409 (local address 127.12.138.193:55779)
0811 02:54:59.551142 (+ 305us) negotiation.cc:107] Waiting for socket to connect
0811 02:54:59.551213 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.138.254:45409: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":222,"thread_start_us":102,"threads_started":1}
I20260811 02:54:59.717450 30873 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:54:59.716874 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.138.254:45409 (local address 127.12.138.194:41247)
0811 02:54:59.717275 (+ 401us) negotiation.cc:107] Waiting for socket to connect
0811 02:54:59.717344 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.138.254:45409: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":241,"thread_start_us":94,"threads_started":1}
I20260811 02:54:59.914068 30874 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:54:59.913636 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.138.254:45409 (local address 127.12.138.195:57951)
0811 02:54:59.913914 (+ 278us) negotiation.cc:107] Waiting for socket to connect
0811 02:54:59.913972 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.138.254:45409: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":206,"thread_start_us":87,"threads_started":1}
I20260811 02:55:00.552595 30875 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:55:00.552167 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.138.254:45409 (local address 127.12.138.193:33787)
0811 02:55:00.552438 (+ 271us) negotiation.cc:107] Waiting for socket to connect
0811 02:55:00.552500 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.138.254:45409: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":198,"thread_start_us":81,"threads_started":1}
I20260811 02:55:00.718753 30876 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:55:00.718306 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.138.254:45409 (local address 127.12.138.194:37967)
0811 02:55:00.718587 (+ 281us) negotiation.cc:107] Waiting for socket to connect
0811 02:55:00.718652 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.138.254:45409: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":198,"thread_start_us":88,"threads_started":1}
I20260811 02:55:00.915524 30878 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:55:00.915069 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.138.254:45409 (local address 127.12.138.195:60941)
0811 02:55:00.915356 (+ 287us) negotiation.cc:107] Waiting for socket to connect
0811 02:55:00.915423 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.138.254:45409: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":196,"thread_start_us":82,"threads_started":1}
I20260811 02:55:01.554210 30879 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:55:01.553787 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.138.254:45409 (local address 127.12.138.193:38163)
0811 02:55:01.554054 (+ 267us) negotiation.cc:107] Waiting for socket to connect
0811 02:55:01.554112 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.138.254:45409: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":175,"thread_start_us":78,"threads_started":1}
I20260811 02:55:01.642551 30880 raft_consensus.cc:482] T 89669c29a2d34fa98ad0180c930e32dd P 13a904a6ad6a46a78dd222ee22eaea3c [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 02:55:01.642774 30880 raft_consensus.cc:504] T 89669c29a2d34fa98ad0180c930e32dd P 13a904a6ad6a46a78dd222ee22eaea3c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "11f70dbb696346a491c8b80eaf7ce67f" member_type: VOTER last_known_addr { host: "127.12.138.195" port: 46749 } } peers { permanent_uuid: "8a43edab73754e09a4b5e387d083f139" member_type: VOTER last_known_addr { host: "127.12.138.194" port: 45119 } } peers { permanent_uuid: "13a904a6ad6a46a78dd222ee22eaea3c" member_type: VOTER last_known_addr { host: "127.12.138.193" port: 40085 } }
I20260811 02:55:01.643354 30880 leader_election.cc:288] T 89669c29a2d34fa98ad0180c930e32dd P 13a904a6ad6a46a78dd222ee22eaea3c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 11f70dbb696346a491c8b80eaf7ce67f (127.12.138.195:46749), 8a43edab73754e09a4b5e387d083f139 (127.12.138.194:45119)
W20260811 02:55:01.644209 30374 leader_election.cc:341] T 89669c29a2d34fa98ad0180c930e32dd P 13a904a6ad6a46a78dd222ee22eaea3c [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 11f70dbb696346a491c8b80eaf7ce67f (127.12.138.195:46749): Not found: Tablet not found: 89669c29a2d34fa98ad0180c930e32dd
W20260811 02:55:01.644356 30373 leader_election.cc:341] T 89669c29a2d34fa98ad0180c930e32dd P 13a904a6ad6a46a78dd222ee22eaea3c [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 8a43edab73754e09a4b5e387d083f139 (127.12.138.194:45119): Not found: Tablet not found: 89669c29a2d34fa98ad0180c930e32dd
I20260811 02:55:01.644454 30373 leader_election.cc:302] T 89669c29a2d34fa98ad0180c930e32dd P 13a904a6ad6a46a78dd222ee22eaea3c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 13a904a6ad6a46a78dd222ee22eaea3c; no voters: 11f70dbb696346a491c8b80eaf7ce67f, 8a43edab73754e09a4b5e387d083f139
I20260811 02:55:01.644639 30880 raft_consensus.cc:2726] T 89669c29a2d34fa98ad0180c930e32dd P 13a904a6ad6a46a78dd222ee22eaea3c [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260811 02:55:01.720343 30881 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:55:01.719876 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.138.254:45409 (local address 127.12.138.194:57781)
0811 02:55:01.720164 (+ 288us) negotiation.cc:107] Waiting for socket to connect
0811 02:55:01.720225 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.138.254:45409: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":187,"thread_start_us":83,"threads_started":1}
I20260811 02:55:01.899748 12843 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDawEwS/build/debug/bin/kudu with pid 30312
I20260811 02:55:01.911286 12843 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDawEwS/build/debug/bin/kudu with pid 30516
I20260811 02:55:01.917069 30882 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:55:01.916501 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.138.254:45409 (local address 127.12.138.195:48485)
0811 02:55:01.916826 (+ 325us) negotiation.cc:107] Waiting for socket to connect
0811 02:55:01.916912 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.138.254:45409: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":205,"thread_start_us":68,"threads_started":1}
I20260811 02:55:01.920495 12843 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDawEwS/build/debug/bin/kudu with pid 30669
2026-08-11T02:55:01Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedIPKICert (19800 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedTSK
Loading random data
Initializing database '/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786416707705494-12843-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 02:55:01 dist-test-slave-9gcw krb5kdc[30885](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 02:55:01 dist-test-slave-9gcw krb5kdc[30885](info): set up 2 sockets
Aug 11 02:55:01 dist-test-slave-9gcw krb5kdc[30885](info): commencing operation
krb5kdc: starting...
W20260811 02:55:03.150243 12843 mini_kdc.cc:121] Time spent starting KDC: real 1.203s user 0.004s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786416707705494-12843-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786416707705494-12843-0/krb5kdc/test-user.keytab.
Aug 11 02:55:03 dist-test-slave-9gcw krb5kdc[30885](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416903, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T02:55:03Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T02:55:03Z Disabled control of system clock
WARNING: no policy specified for kudu/127.12.138.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.138.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.138.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786416707705494-12843-0/krb5kdc/kudu_127.12.138.254.keytab.
Entry for principal kudu/127.12.138.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786416707705494-12843-0/krb5kdc/kudu_127.12.138.254.keytab.
I20260811 02:55:03.294175 12843 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDawEwS/build/debug/bin/kudu
/tmp/dist-test-taskDawEwS/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786416707705494-12843-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786416707705494-12843-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786416707705494-12843-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786416707705494-12843-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.138.254:39391
--webserver_interface=127.12.138.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786416707705494-12843-0/krb5kdc/kudu_127.12.138.254.keytab
--principal=kudu/127.12.138.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.138.212:42413
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.12.138.254:39391
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786416707705494-12843-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786416707705494-12843-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786416707705494-12843-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 02:55:03.420384 30899 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 02:55:03.420684 30899 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 02:55:03.420729 30899 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 02:55:03.424925 30899 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260811 02:55:03.424995 30899 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260811 02:55:03.425015 30899 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260811 02:55:03.425032 30899 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 02:55:03.425102 30899 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 02:55:03.425169 30899 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 02:55:03.425209 30899 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 02:55:03.425252 30899 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 02:55:03.430698 30899 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.138.212:42413
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786416707705494-12843-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786416707705494-12843-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.12.138.254:39391
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786416707705494-12843-0/krb5kdc/kudu_127.12.138.254.keytab
--principal=kudu/127.12.138.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.138.254:39391
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786416707705494-12843-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.12.138.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30899
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786416707705494-12843-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 02:09:49 UTC on ccycloud.quasar-fsnqqc.root.comops.site
build id 4579570
I20260811 02:55:03.431988 30899 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 02:55:03.433365 30899 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 02:55:03.443197 30909 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:55:03.443207 30905 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:55:03.443302 30899 server_base.cc:1034] running on GCE node
W20260811 02:55:03.443218 30906 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:55:03.444129 30899 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 02:55:03.445649 30899 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 02:55:03.446866 30899 hybrid_clock.cc:648] HybridClock initialized: now 1786416903446841 us; error 37 us; skew 500 ppm
Aug 11 02:55:03 dist-test-slave-9gcw krb5kdc[30885](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416903, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.138.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 02:55:03.455183 30899 init.cc:377] Logged in from keytab as kudu/127.12.138.254@KRBTEST.COM (short username kudu)
I20260811 02:55:03.456244 30899 webserver.cc:460] Webserver started at http://127.12.138.254:44325/ using document root <none> and password file <none>
I20260811 02:55:03.456875 30899 fs_manager.cc:359] Metadata directory not provided
I20260811 02:55:03.456976 30899 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 02:55:03.457298 30899 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 02:55:03.459913 30899 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786416707705494-12843-0/minicluster-data/master-0/data/instance:
uuid: "4ce22d68cbc247349baf03912c7e6cb6"
format_stamp: "Formatted at 2026-08-11 02:55:03 on dist-test-slave-9gcw"
server_key: "82b9b2173efd9043c9a0a661c2784363"
server_key_iv: "59f89185dfc326077347b28838f76d15"
server_key_version: "encryptionkey@0"
I20260811 02:55:03.460515 30899 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786416707705494-12843-0/minicluster-data/master-0/wal/instance:
uuid: "4ce22d68cbc247349baf03912c7e6cb6"
format_stamp: "Formatted at 2026-08-11 02:55:03 on dist-test-slave-9gcw"
server_key: "82b9b2173efd9043c9a0a661c2784363"
server_key_iv: "59f89185dfc326077347b28838f76d15"
server_key_version: "encryptionkey@0"
I20260811 02:55:03.464655 30899 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260811 02:55:03.467473 30916 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 02:55:03.468874 30899 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260811 02:55:03.469045 30899 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786416707705494-12843-0/minicluster-data/master-0/data,/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786416707705494-12843-0/minicluster-data/master-0/wal
uuid: "4ce22d68cbc247349baf03912c7e6cb6"
format_stamp: "Formatted at 2026-08-11 02:55:03 on dist-test-slave-9gcw"
server_key: "82b9b2173efd9043c9a0a661c2784363"
server_key_iv: "59f89185dfc326077347b28838f76d15"
server_key_version: "encryptionkey@0"
I20260811 02:55:03.469197 30899 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786416707705494-12843-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786416707705494-12843-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786416707705494-12843-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 02:55:03.487408 30899 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 02:55:03.490717 30899 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 02:55:03.490949 30899 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 02:55:03.498386 30899 rpc_server.cc:307] RPC server started. Bound to: 127.12.138.254:39391
I20260811 02:55:03.498778 30976 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.138.254:39391 every 8 connection(s)
I20260811 02:55:03.499450 30899 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786416707705494-12843-0/minicluster-data/master-0/data/info.pb
I20260811 02:55:03.502580 30977 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 02:55:03.502897 12843 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDawEwS/build/debug/bin/kudu as pid 30899
I20260811 02:55:03.503046 12843 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786416707705494-12843-0/minicluster-data/master-0/wal/instance
I20260811 02:55:03.503371 12843 external_mini_cluster.cc:1428] Setting key a893983d14d7ba69e38a8c4be8526949
I20260811 02:55:03.510015 30977 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4ce22d68cbc247349baf03912c7e6cb6: Bootstrap starting.
I20260811 02:55:03.512300 30977 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4ce22d68cbc247349baf03912c7e6cb6: Neither blocks nor log segments found. Creating new log.
Aug 11 02:55:03 dist-test-slave-9gcw krb5kdc[30885](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416903, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.12.138.254@KRBTEST.COM
I20260811 02:55:03.513216 30977 log.cc:824] T 00000000000000000000000000000000 P 4ce22d68cbc247349baf03912c7e6cb6: Log is configured to *not* fsync() on all Append() calls
I20260811 02:55:03.515556 30977 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4ce22d68cbc247349baf03912c7e6cb6: No bootstrap required, opened a new log
I20260811 02:55:03.518102 30980 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:55:03.504817 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34984 (local address 127.12.138.254:39391)
0811 02:55:03.505305 (+ 488us) server_negotiation.cc:206] Beginning negotiation
0811 02:55:03.505328 (+ 23us) server_negotiation.cc:399] Waiting for connection header
0811 02:55:03.505374 (+ 46us) server_negotiation.cc:407] Connection header received
0811 02:55:03.506311 (+ 937us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:55:03.506337 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:55:03.506799 (+ 462us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:55:03.507197 (+ 398us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:55:03.507944 (+ 747us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:55:03.508953 (+ 1009us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:55:03.509909 (+ 956us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:55:03.510437 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:55:03.510631 (+ 194us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:55:03.513239 (+ 2608us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:55:03.513244 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:55:03.513257 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:55:03.513285 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:55:03.515161 (+ 1876us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:55:03.515642 (+ 481us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:55:03.515648 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:55:03.515652 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:55:03.515813 (+ 161us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:55:03.516099 (+ 286us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:55:03.516103 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:55:03.516105 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:55:03.516504 (+ 399us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:55:03.516710 (+ 206us) server_negotiation.cc:1035] Waiting for connection context
0811 02:55:03.516993 (+ 283us) server_negotiation.cc:299] Negotiation successful
0811 02:55:03.517331 (+ 338us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":333,"thread_start_us":126,"threads_started":1}
I20260811 02:55:03.519450 30977 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4ce22d68cbc247349baf03912c7e6cb6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4ce22d68cbc247349baf03912c7e6cb6" member_type: VOTER last_known_addr { host: "127.12.138.254" port: 39391 } }
I20260811 02:55:03.519740 30977 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4ce22d68cbc247349baf03912c7e6cb6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 02:55:03.519806 30977 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4ce22d68cbc247349baf03912c7e6cb6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4ce22d68cbc247349baf03912c7e6cb6, State: Initialized, Role: FOLLOWER
I20260811 02:55:03.520303 30977 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4ce22d68cbc247349baf03912c7e6cb6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4ce22d68cbc247349baf03912c7e6cb6" member_type: VOTER last_known_addr { host: "127.12.138.254" port: 39391 } }
I20260811 02:55:03.520469 30977 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4ce22d68cbc247349baf03912c7e6cb6 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 02:55:03.520540 30977 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4ce22d68cbc247349baf03912c7e6cb6 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 02:55:03.520614 30977 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4ce22d68cbc247349baf03912c7e6cb6 [term 0 FOLLOWER]: Advancing to term 1
I20260811 02:55:03.521584 30977 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4ce22d68cbc247349baf03912c7e6cb6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4ce22d68cbc247349baf03912c7e6cb6" member_type: VOTER last_known_addr { host: "127.12.138.254" port: 39391 } }
I20260811 02:55:03.521967 30977 leader_election.cc:302] T 00000000000000000000000000000000 P 4ce22d68cbc247349baf03912c7e6cb6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4ce22d68cbc247349baf03912c7e6cb6; no voters:
I20260811 02:55:03.522383 30977 leader_election.cc:288] T 00000000000000000000000000000000 P 4ce22d68cbc247349baf03912c7e6cb6 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 02:55:03.522459 30983 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4ce22d68cbc247349baf03912c7e6cb6 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 02:55:03.522697 30983 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4ce22d68cbc247349baf03912c7e6cb6 [term 1 LEADER]: Becoming Leader. State: Replica: 4ce22d68cbc247349baf03912c7e6cb6, State: Running, Role: LEADER
I20260811 02:55:03.522998 30983 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4ce22d68cbc247349baf03912c7e6cb6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4ce22d68cbc247349baf03912c7e6cb6" member_type: VOTER last_known_addr { host: "127.12.138.254" port: 39391 } }
I20260811 02:55:03.523685 30977 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4ce22d68cbc247349baf03912c7e6cb6 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 02:55:03.524849 30984 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4ce22d68cbc247349baf03912c7e6cb6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4ce22d68cbc247349baf03912c7e6cb6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4ce22d68cbc247349baf03912c7e6cb6" member_type: VOTER last_known_addr { host: "127.12.138.254" port: 39391 } } }
I20260811 02:55:03.524857 30985 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4ce22d68cbc247349baf03912c7e6cb6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4ce22d68cbc247349baf03912c7e6cb6. Latest consensus state: current_term: 1 leader_uuid: "4ce22d68cbc247349baf03912c7e6cb6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4ce22d68cbc247349baf03912c7e6cb6" member_type: VOTER last_known_addr { host: "127.12.138.254" port: 39391 } } }
I20260811 02:55:03.525023 30984 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4ce22d68cbc247349baf03912c7e6cb6 [sys.catalog]: This master's current role is: LEADER
I20260811 02:55:03.525063 30985 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4ce22d68cbc247349baf03912c7e6cb6 [sys.catalog]: This master's current role is: LEADER
W20260811 02:55:03.528497 30999 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 4ce22d68cbc247349baf03912c7e6cb6: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 02:55:03.528564 30999 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 02:55:03.528643 30992 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 02:55:03.529419 30992 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 02:55:03.534962 30992 catalog_manager.cc:1347] Generated new cluster ID: ef117953dc34499cbebe471f50cf1592
I20260811 02:55:03.535032 30992 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 02:55:03.579723 30992 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 02:55:03.581005 30992 catalog_manager.cc:1504] Loading token signing keys...
I20260811 02:55:03.588605 30992 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4ce22d68cbc247349baf03912c7e6cb6: Generated new TSK 0
I20260811 02:55:03.589468 30992 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.12.138.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.138.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.138.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786416707705494-12843-0/krb5kdc/kudu_127.12.138.193.keytab.
Entry for principal kudu/127.12.138.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786416707705494-12843-0/krb5kdc/kudu_127.12.138.193.keytab.
I20260811 02:55:03.636893 12843 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDawEwS/build/debug/bin/kudu
/tmp/dist-test-taskDawEwS/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786416707705494-12843-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786416707705494-12843-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786416707705494-12843-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786416707705494-12843-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.138.193:0
--local_ip_for_outbound_sockets=127.12.138.193
--webserver_interface=127.12.138.193
--webserver_port=0
--tserver_master_addrs=127.12.138.254:39391
--keytab_file=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786416707705494-12843-0/krb5kdc/kudu_127.12.138.193.keytab
--principal=kudu/127.12.138.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.138.212:42413
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786416707705494-12843-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786416707705494-12843-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786416707705494-12843-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 02:55:03.753904 31006 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 02:55:03.754179 31006 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 02:55:03.754221 31006 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 02:55:03.758277 31006 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 02:55:03.758353 31006 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 02:55:03.758447 31006 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.138.193
I20260811 02:55:03.763827 31006 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.138.212:42413
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786416707705494-12843-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786416707705494-12843-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786416707705494-12843-0/krb5kdc/kudu_127.12.138.193.keytab
--principal=kudu/127.12.138.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.138.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786416707705494-12843-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.12.138.193
--webserver_port=0
--tserver_master_addrs=127.12.138.254:39391
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31006
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.138.193
--log_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786416707705494-12843-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 02:09:49 UTC on ccycloud.quasar-fsnqqc.root.comops.site
build id 4579570
I20260811 02:55:03.764969 31006 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 02:55:03.766055 31006 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 02:55:03.776407 31012 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:55:03.776437 31011 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:55:03.776407 31014 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:55:03.776791 31006 server_base.cc:1034] running on GCE node
I20260811 02:55:03.777388 31006 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 02:55:03.778106 31006 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 02:55:03.779282 31006 hybrid_clock.cc:648] HybridClock initialized: now 1786416903779256 us; error 61 us; skew 500 ppm
Aug 11 02:55:03 dist-test-slave-9gcw krb5kdc[30885](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416903, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.138.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 02:55:03.782374 31006 init.cc:377] Logged in from keytab as kudu/127.12.138.193@KRBTEST.COM (short username kudu)
I20260811 02:55:03.783352 31006 webserver.cc:460] Webserver started at http://127.12.138.193:41777/ using document root <none> and password file <none>
I20260811 02:55:03.784049 31006 fs_manager.cc:359] Metadata directory not provided
I20260811 02:55:03.784158 31006 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 02:55:03.784410 31006 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 02:55:03.786636 31006 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786416707705494-12843-0/minicluster-data/ts-0/data/instance:
uuid: "b276426cf6d74110ac231a84ac3a3778"
format_stamp: "Formatted at 2026-08-11 02:55:03 on dist-test-slave-9gcw"
server_key: "4705960a372f5dbbafe86ae8d0016d6a"
server_key_iv: "c017c9d055ebf8a2f24ec71073419281"
server_key_version: "encryptionkey@0"
I20260811 02:55:03.787257 31006 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786416707705494-12843-0/minicluster-data/ts-0/wal/instance:
uuid: "b276426cf6d74110ac231a84ac3a3778"
format_stamp: "Formatted at 2026-08-11 02:55:03 on dist-test-slave-9gcw"
server_key: "4705960a372f5dbbafe86ae8d0016d6a"
server_key_iv: "c017c9d055ebf8a2f24ec71073419281"
server_key_version: "encryptionkey@0"
I20260811 02:55:03.791226 31006 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260811 02:55:03.793946 31023 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 02:55:03.795246 31006 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260811 02:55:03.795409 31006 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786416707705494-12843-0/minicluster-data/ts-0/data,/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786416707705494-12843-0/minicluster-data/ts-0/wal
uuid: "b276426cf6d74110ac231a84ac3a3778"
format_stamp: "Formatted at 2026-08-11 02:55:03 on dist-test-slave-9gcw"
server_key: "4705960a372f5dbbafe86ae8d0016d6a"
server_key_iv: "c017c9d055ebf8a2f24ec71073419281"
server_key_version: "encryptionkey@0"
I20260811 02:55:03.795535 31006 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786416707705494-12843-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786416707705494-12843-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786416707705494-12843-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 02:55:03.819031 31006 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 02:55:03.822763 31006 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 02:55:03.822992 31006 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 02:55:03.823689 31006 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 02:55:03.824719 31006 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 02:55:03.824800 31006 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 02:55:03.824879 31006 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 02:55:03.824923 31006 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 02:55:03.835116 31006 rpc_server.cc:307] RPC server started. Bound to: 127.12.138.193:41093
I20260811 02:55:03.835237 31151 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.138.193:41093 every 8 connection(s)
I20260811 02:55:03.836153 31006 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786416707705494-12843-0/minicluster-data/ts-0/data/info.pb
I20260811 02:55:03.845258 12843 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDawEwS/build/debug/bin/kudu as pid 31006
I20260811 02:55:03.845436 12843 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786416707705494-12843-0/minicluster-data/ts-0/wal/instance
I20260811 02:55:03.845780 12843 external_mini_cluster.cc:1428] Setting key 6d2fbc201d05779185c240c2fa2b4740
Aug 11 02:55:03 dist-test-slave-9gcw krb5kdc[30885](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416903, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.138.193@KRBTEST.COM for kudu/127.12.138.254@KRBTEST.COM
I20260811 02:55:03.851503 30980 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:55:03.838344 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.138.193:42121 (local address 127.12.138.254:39391)
0811 02:55:03.838537 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0811 02:55:03.838542 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 02:55:03.839669 (+ 1127us) server_negotiation.cc:407] Connection header received
0811 02:55:03.840637 (+ 968us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:55:03.840643 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:55:03.840700 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:55:03.840815 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:55:03.842199 (+ 1384us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:55:03.842749 (+ 550us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:55:03.843964 (+ 1215us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:55:03.844193 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:55:03.844273 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:55:03.847203 (+ 2930us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:55:03.847207 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:55:03.847211 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:55:03.847241 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:55:03.849092 (+ 1851us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:55:03.849720 (+ 628us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:55:03.849726 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:55:03.849729 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:55:03.849844 (+ 115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:55:03.850313 (+ 469us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:55:03.850316 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:55:03.850317 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:55:03.850496 (+ 179us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:55:03.850573 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0811 02:55:03.851266 (+ 693us) server_negotiation.cc:299] Negotiation successful
0811 02:55:03.851381 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":83}
I20260811 02:55:03.852746 31154 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:55:03.838736 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.138.254:39391 (local address 127.12.138.193:42121)
0811 02:55:03.839480 (+ 744us) negotiation.cc:107] Waiting for socket to connect
0811 02:55:03.839536 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0811 02:55:03.840402 (+ 866us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:55:03.840992 (+ 590us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:55:03.840999 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:55:03.841454 (+ 455us) client_negotiation.cc:189] Negotiated authn=SASL
0811 02:55:03.842052 (+ 598us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:55:03.842067 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:55:03.842875 (+ 808us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:55:03.842878 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:55:03.843852 (+ 974us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:55:03.843862 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:55:03.844328 (+ 466us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:55:03.844331 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:55:03.844481 (+ 150us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:55:03.845231 (+ 750us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 02:55:03.845258 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0811 02:55:03.847032 (+ 1774us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 02:55:03.849288 (+ 2256us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:55:03.849293 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:55:03.849306 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:55:03.849585 (+ 279us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:55:03.849955 (+ 370us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:55:03.849961 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:55:03.849965 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:55:03.850206 (+ 241us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:55:03.850603 (+ 397us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 02:55:03.850611 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 02:55:03.850972 (+ 361us) client_negotiation.cc:769] Sending connection context
0811 02:55:03.851224 (+ 252us) client_negotiation.cc:240] Negotiation successful
0811 02:55:03.851600 (+ 376us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":540,"thread_start_us":143,"threads_started":1}
I20260811 02:55:03.854218 31152 heartbeater.cc:347] Connected to a master server at 127.12.138.254:39391
I20260811 02:55:03.854501 31152 heartbeater.cc:464] Registering TS with master...
I20260811 02:55:03.855031 31152 heartbeater.cc:511] Master 127.12.138.254:39391 requested a full tablet report, sending...
I20260811 02:55:03.856671 30935 ts_manager.cc:194] Registered new tserver with Master: b276426cf6d74110ac231a84ac3a3778 (127.12.138.193:41093)
WARNING: no policy specified for kudu/127.12.138.194@KRBTEST.COM; defaulting to no policy
I20260811 02:55:03.857997 30935 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.12.138.193@KRBTEST.COM'} at 127.12.138.193:42121
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.138.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.138.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786416707705494-12843-0/krb5kdc/kudu_127.12.138.194.keytab.
Entry for principal kudu/127.12.138.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786416707705494-12843-0/krb5kdc/kudu_127.12.138.194.keytab.
I20260811 02:55:03.880992 12843 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDawEwS/build/debug/bin/kudu
/tmp/dist-test-taskDawEwS/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786416707705494-12843-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786416707705494-12843-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786416707705494-12843-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786416707705494-12843-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.138.194:0
--local_ip_for_outbound_sockets=127.12.138.194
--webserver_interface=127.12.138.194
--webserver_port=0
--tserver_master_addrs=127.12.138.254:39391
--keytab_file=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786416707705494-12843-0/krb5kdc/kudu_127.12.138.194.keytab
--principal=kudu/127.12.138.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.138.212:42413
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786416707705494-12843-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786416707705494-12843-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786416707705494-12843-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 02:55:03.998709 31157 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 02:55:03.999001 31157 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 02:55:03.999083 31157 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 02:55:04.003414 31157 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 02:55:04.003504 31157 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 02:55:04.003609 31157 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.138.194
I20260811 02:55:04.009033 31157 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.138.212:42413
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786416707705494-12843-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786416707705494-12843-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786416707705494-12843-0/krb5kdc/kudu_127.12.138.194.keytab
--principal=kudu/127.12.138.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.138.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786416707705494-12843-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.12.138.194
--webserver_port=0
--tserver_master_addrs=127.12.138.254:39391
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31157
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.138.194
--log_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786416707705494-12843-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 02:09:49 UTC on ccycloud.quasar-fsnqqc.root.comops.site
build id 4579570
I20260811 02:55:04.010231 31157 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 02:55:04.011297 31157 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 02:55:04.021492 31162 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:55:04.021664 31157 server_base.cc:1034] running on GCE node
W20260811 02:55:04.021524 31165 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:55:04.021498 31163 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:55:04.022208 31157 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 02:55:04.022872 31157 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 02:55:04.024073 31157 hybrid_clock.cc:648] HybridClock initialized: now 1786416904024034 us; error 53 us; skew 500 ppm
Aug 11 02:55:04 dist-test-slave-9gcw krb5kdc[30885](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416904, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.138.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 02:55:04.027158 31157 init.cc:377] Logged in from keytab as kudu/127.12.138.194@KRBTEST.COM (short username kudu)
I20260811 02:55:04.028177 31157 webserver.cc:460] Webserver started at http://127.12.138.194:33595/ using document root <none> and password file <none>
I20260811 02:55:04.028780 31157 fs_manager.cc:359] Metadata directory not provided
I20260811 02:55:04.028875 31157 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 02:55:04.029127 31157 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 02:55:04.031409 31157 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786416707705494-12843-0/minicluster-data/ts-1/data/instance:
uuid: "93b8a8a73e364049893bd06522ab3d7c"
format_stamp: "Formatted at 2026-08-11 02:55:04 on dist-test-slave-9gcw"
server_key: "adb77886557a128a3658d3768cfeb27d"
server_key_iv: "73633ad8d01bc30fbc2dc442c7d12812"
server_key_version: "encryptionkey@0"
I20260811 02:55:04.031996 31157 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786416707705494-12843-0/minicluster-data/ts-1/wal/instance:
uuid: "93b8a8a73e364049893bd06522ab3d7c"
format_stamp: "Formatted at 2026-08-11 02:55:04 on dist-test-slave-9gcw"
server_key: "adb77886557a128a3658d3768cfeb27d"
server_key_iv: "73633ad8d01bc30fbc2dc442c7d12812"
server_key_version: "encryptionkey@0"
I20260811 02:55:04.035950 31157 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260811 02:55:04.038769 31174 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 02:55:04.040278 31157 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260811 02:55:04.040534 31157 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786416707705494-12843-0/minicluster-data/ts-1/data,/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786416707705494-12843-0/minicluster-data/ts-1/wal
uuid: "93b8a8a73e364049893bd06522ab3d7c"
format_stamp: "Formatted at 2026-08-11 02:55:04 on dist-test-slave-9gcw"
server_key: "adb77886557a128a3658d3768cfeb27d"
server_key_iv: "73633ad8d01bc30fbc2dc442c7d12812"
server_key_version: "encryptionkey@0"
I20260811 02:55:04.040668 31157 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786416707705494-12843-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786416707705494-12843-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786416707705494-12843-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 02:55:04.051164 31157 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 02:55:04.054730 31157 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 02:55:04.054971 31157 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 02:55:04.055613 31157 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 02:55:04.056484 31157 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 02:55:04.056558 31157 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 02:55:04.056632 31157 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 02:55:04.056675 31157 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 02:55:04.065779 31157 rpc_server.cc:307] RPC server started. Bound to: 127.12.138.194:42231
I20260811 02:55:04.065811 31305 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.138.194:42231 every 8 connection(s)
I20260811 02:55:04.066725 31157 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786416707705494-12843-0/minicluster-data/ts-1/data/info.pb
I20260811 02:55:04.068745 12843 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDawEwS/build/debug/bin/kudu as pid 31157
I20260811 02:55:04.068892 12843 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786416707705494-12843-0/minicluster-data/ts-1/wal/instance
I20260811 02:55:04.069270 12843 external_mini_cluster.cc:1428] Setting key 879d52ac7f5038a01c72f95ca6d49857
Aug 11 02:55:04 dist-test-slave-9gcw krb5kdc[30885](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416904, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.138.194@KRBTEST.COM for kudu/127.12.138.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.12.138.195@KRBTEST.COM; defaulting to no policy
I20260811 02:55:04.081918 30980 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:55:04.068653 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.138.194:52443 (local address 127.12.138.254:39391)
0811 02:55:04.068831 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0811 02:55:04.068836 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 02:55:04.070134 (+ 1298us) server_negotiation.cc:407] Connection header received
0811 02:55:04.071016 (+ 882us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:55:04.071019 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:55:04.071075 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:55:04.071162 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:55:04.072350 (+ 1188us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:55:04.073263 (+ 913us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:55:04.074601 (+ 1338us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:55:04.074865 (+ 264us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:55:04.074952 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:55:04.078076 (+ 3124us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:55:04.078079 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:55:04.078082 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:55:04.078114 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:55:04.079666 (+ 1552us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:55:04.080225 (+ 559us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:55:04.080229 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:55:04.080230 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:55:04.080314 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:55:04.080790 (+ 476us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:55:04.080792 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:55:04.080793 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:55:04.080983 (+ 190us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:55:04.081063 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0811 02:55:04.081650 (+ 587us) server_negotiation.cc:299] Negotiation successful
0811 02:55:04.081790 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260811 02:55:04.082764 31308 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:55:04.069183 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.138.254:39391 (local address 127.12.138.194:52443)
0811 02:55:04.069968 (+ 785us) negotiation.cc:107] Waiting for socket to connect
0811 02:55:04.070021 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0811 02:55:04.070820 (+ 799us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:55:04.071313 (+ 493us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:55:04.071319 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:55:04.071668 (+ 349us) client_negotiation.cc:189] Negotiated authn=SASL
0811 02:55:04.072197 (+ 529us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:55:04.072211 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:55:04.073414 (+ 1203us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:55:04.073418 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:55:04.074479 (+ 1061us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:55:04.074494 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:55:04.075028 (+ 534us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:55:04.075034 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:55:04.075272 (+ 238us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:55:04.075967 (+ 695us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 02:55:04.075997 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0811 02:55:04.077893 (+ 1896us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 02:55:04.079825 (+ 1932us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:55:04.079831 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:55:04.079847 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:55:04.080113 (+ 266us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:55:04.080450 (+ 337us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:55:04.080455 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:55:04.080458 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:55:04.080652 (+ 194us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:55:04.081082 (+ 430us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 02:55:04.081087 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 02:55:04.081393 (+ 306us) client_negotiation.cc:769] Sending connection context
0811 02:55:04.081602 (+ 209us) client_negotiation.cc:240] Negotiation successful
0811 02:55:04.081885 (+ 283us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":569,"thread_start_us":160,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.138.195@KRBTEST.COM" created.
I20260811 02:55:04.084200 31306 heartbeater.cc:347] Connected to a master server at 127.12.138.254:39391
I20260811 02:55:04.084477 31306 heartbeater.cc:464] Registering TS with master...
I20260811 02:55:04.085011 31306 heartbeater.cc:511] Master 127.12.138.254:39391 requested a full tablet report, sending...
I20260811 02:55:04.086211 30935 ts_manager.cc:194] Registered new tserver with Master: 93b8a8a73e364049893bd06522ab3d7c (127.12.138.194:42231)
I20260811 02:55:04.086833 30935 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.12.138.194@KRBTEST.COM'} at 127.12.138.194:52443
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.138.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786416707705494-12843-0/krb5kdc/kudu_127.12.138.195.keytab.
Entry for principal kudu/127.12.138.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786416707705494-12843-0/krb5kdc/kudu_127.12.138.195.keytab.
I20260811 02:55:04.104208 12843 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDawEwS/build/debug/bin/kudu
/tmp/dist-test-taskDawEwS/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786416707705494-12843-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786416707705494-12843-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786416707705494-12843-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786416707705494-12843-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.138.195:0
--local_ip_for_outbound_sockets=127.12.138.195
--webserver_interface=127.12.138.195
--webserver_port=0
--tserver_master_addrs=127.12.138.254:39391
--keytab_file=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786416707705494-12843-0/krb5kdc/kudu_127.12.138.195.keytab
--principal=kudu/127.12.138.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.138.212:42413
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786416707705494-12843-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786416707705494-12843-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786416707705494-12843-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 02:55:04.230577 31311 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 02:55:04.230862 31311 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 02:55:04.230907 31311 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 02:55:04.234948 31311 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 02:55:04.235011 31311 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 02:55:04.235070 31311 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.138.195
I20260811 02:55:04.240422 31311 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.138.212:42413
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786416707705494-12843-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786416707705494-12843-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786416707705494-12843-0/krb5kdc/kudu_127.12.138.195.keytab
--principal=kudu/127.12.138.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.138.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786416707705494-12843-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.12.138.195
--webserver_port=0
--tserver_master_addrs=127.12.138.254:39391
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31311
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.138.195
--log_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786416707705494-12843-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 02:09:49 UTC on ccycloud.quasar-fsnqqc.root.comops.site
build id 4579570
I20260811 02:55:04.241521 31311 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 02:55:04.242574 31311 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 02:55:04.252857 31317 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:55:04.252857 31318 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:55:04.252939 31321 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:55:04.253216 31311 server_base.cc:1034] running on GCE node
I20260811 02:55:04.253643 31311 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 02:55:04.254266 31311 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 02:55:04.255461 31311 hybrid_clock.cc:648] HybridClock initialized: now 1786416904255425 us; error 45 us; skew 500 ppm
Aug 11 02:55:04 dist-test-slave-9gcw krb5kdc[30885](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416904, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.138.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 02:55:04.258462 31311 init.cc:377] Logged in from keytab as kudu/127.12.138.195@KRBTEST.COM (short username kudu)
I20260811 02:55:04.259445 31311 webserver.cc:460] Webserver started at http://127.12.138.195:37877/ using document root <none> and password file <none>
I20260811 02:55:04.260042 31311 fs_manager.cc:359] Metadata directory not provided
I20260811 02:55:04.260135 31311 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 02:55:04.260389 31311 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 02:55:04.262674 31311 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786416707705494-12843-0/minicluster-data/ts-2/data/instance:
uuid: "924ad6fccb5f44c6a9da839c8b730081"
format_stamp: "Formatted at 2026-08-11 02:55:04 on dist-test-slave-9gcw"
server_key: "a63d3bf5d003f94c5a55c3705422df2e"
server_key_iv: "436cefcc8a36da9bf0941d370ca47a77"
server_key_version: "encryptionkey@0"
I20260811 02:55:04.263267 31311 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786416707705494-12843-0/minicluster-data/ts-2/wal/instance:
uuid: "924ad6fccb5f44c6a9da839c8b730081"
format_stamp: "Formatted at 2026-08-11 02:55:04 on dist-test-slave-9gcw"
server_key: "a63d3bf5d003f94c5a55c3705422df2e"
server_key_iv: "436cefcc8a36da9bf0941d370ca47a77"
server_key_version: "encryptionkey@0"
I20260811 02:55:04.267593 31311 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260811 02:55:04.270360 31331 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 02:55:04.271797 31311 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260811 02:55:04.271956 31311 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786416707705494-12843-0/minicluster-data/ts-2/data,/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786416707705494-12843-0/minicluster-data/ts-2/wal
uuid: "924ad6fccb5f44c6a9da839c8b730081"
format_stamp: "Formatted at 2026-08-11 02:55:04 on dist-test-slave-9gcw"
server_key: "a63d3bf5d003f94c5a55c3705422df2e"
server_key_iv: "436cefcc8a36da9bf0941d370ca47a77"
server_key_version: "encryptionkey@0"
I20260811 02:55:04.272084 31311 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786416707705494-12843-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786416707705494-12843-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786416707705494-12843-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 02:55:04.301556 31311 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 02:55:04.305274 31311 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 02:55:04.305503 31311 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 02:55:04.306150 31311 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 02:55:04.307041 31311 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 02:55:04.307114 31311 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 02:55:04.307181 31311 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 02:55:04.307224 31311 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 02:55:04.316959 31311 rpc_server.cc:307] RPC server started. Bound to: 127.12.138.195:35627
I20260811 02:55:04.316982 31466 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.138.195:35627 every 8 connection(s)
I20260811 02:55:04.318037 31311 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786416707705494-12843-0/minicluster-data/ts-2/data/info.pb
I20260811 02:55:04.322800 12843 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDawEwS/build/debug/bin/kudu as pid 31311
I20260811 02:55:04.322956 12843 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786416707705494-12843-0/minicluster-data/ts-2/wal/instance
I20260811 02:55:04.323323 12843 external_mini_cluster.cc:1428] Setting key 8c1711dffa29d366707fe95a7e08f504
Aug 11 02:55:04 dist-test-slave-9gcw krb5kdc[30885](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416904, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.138.195@KRBTEST.COM for kudu/127.12.138.254@KRBTEST.COM
I20260811 02:55:04.332448 30980 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:55:04.319953 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.138.195:57837 (local address 127.12.138.254:39391)
0811 02:55:04.320157 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0811 02:55:04.320161 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 02:55:04.321306 (+ 1145us) server_negotiation.cc:407] Connection header received
0811 02:55:04.322289 (+ 983us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:55:04.322303 (+ 14us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:55:04.322351 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:55:04.322429 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:55:04.324207 (+ 1778us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:55:04.324964 (+ 757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:55:04.326466 (+ 1502us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:55:04.326681 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:55:04.326762 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:55:04.329408 (+ 2646us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:55:04.329411 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:55:04.329413 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:55:04.329437 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:55:04.330697 (+ 1260us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:55:04.331159 (+ 462us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:55:04.331162 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:55:04.331163 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:55:04.331235 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:55:04.331537 (+ 302us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:55:04.331539 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:55:04.331540 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:55:04.331703 (+ 163us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:55:04.331776 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0811 02:55:04.332215 (+ 439us) server_negotiation.cc:299] Negotiation successful
0811 02:55:04.332328 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260811 02:55:04.333468 31470 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:55:04.320356 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.138.254:39391 (local address 127.12.138.195:57837)
0811 02:55:04.321079 (+ 723us) negotiation.cc:107] Waiting for socket to connect
0811 02:55:04.321123 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0811 02:55:04.322079 (+ 956us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:55:04.322656 (+ 577us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:55:04.322667 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:55:04.323282 (+ 615us) client_negotiation.cc:189] Negotiated authn=SASL
0811 02:55:04.324037 (+ 755us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:55:04.324057 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:55:04.325124 (+ 1067us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:55:04.325128 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:55:04.326354 (+ 1226us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:55:04.326364 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:55:04.326837 (+ 473us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:55:04.326839 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:55:04.326983 (+ 144us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:55:04.327632 (+ 649us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 02:55:04.327650 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0811 02:55:04.329249 (+ 1599us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 02:55:04.330801 (+ 1552us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:55:04.330806 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:55:04.330818 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:55:04.331067 (+ 249us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:55:04.331310 (+ 243us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:55:04.331312 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:55:04.331313 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:55:04.331454 (+ 141us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:55:04.331784 (+ 330us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 02:55:04.331789 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 02:55:04.332009 (+ 220us) client_negotiation.cc:769] Sending connection context
0811 02:55:04.332158 (+ 149us) client_negotiation.cc:240] Negotiation successful
0811 02:55:04.332412 (+ 254us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":525,"thread_start_us":117,"threads_started":1}
I20260811 02:55:04.334564 31467 heartbeater.cc:347] Connected to a master server at 127.12.138.254:39391
I20260811 02:55:04.334824 31467 heartbeater.cc:464] Registering TS with master...
I20260811 02:55:04.335309 31467 heartbeater.cc:511] Master 127.12.138.254:39391 requested a full tablet report, sending...
I20260811 02:55:04.336361 30935 ts_manager.cc:194] Registered new tserver with Master: 924ad6fccb5f44c6a9da839c8b730081 (127.12.138.195:35627)
I20260811 02:55:04.337070 30935 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.12.138.195@KRBTEST.COM'} at 127.12.138.195:57837
I20260811 02:55:04.337713 12843 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 02:55:04.347084 30980 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:55:04.339690 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34990 (local address 127.12.138.254:39391)
0811 02:55:04.339810 (+ 120us) server_negotiation.cc:206] Beginning negotiation
0811 02:55:04.339814 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 02:55:04.340026 (+ 212us) server_negotiation.cc:407] Connection header received
0811 02:55:04.340115 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:55:04.340117 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:55:04.340152 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:55:04.340216 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:55:04.340940 (+ 724us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:55:04.341528 (+ 588us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:55:04.342364 (+ 836us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:55:04.342564 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:55:04.342637 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:55:04.343990 (+ 1353us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:55:04.343993 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:55:04.343995 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:55:04.344012 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:55:04.345541 (+ 1529us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:55:04.345951 (+ 410us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:55:04.345953 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:55:04.345954 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:55:04.346025 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:55:04.346311 (+ 286us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:55:04.346313 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:55:04.346314 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:55:04.346519 (+ 205us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:55:04.346621 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0811 02:55:04.346882 (+ 261us) server_negotiation.cc:299] Negotiation successful
0811 02:55:04.346971 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260811 02:55:04.350127 30933 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34990:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 02:55:04.353124 30933 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 02:55:04.367928 31481 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:55:04.361928 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.138.195:35627 (local address 127.0.0.1:42896)
0811 02:55:04.362455 (+ 527us) negotiation.cc:107] Waiting for socket to connect
0811 02:55:04.362505 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0811 02:55:04.362660 (+ 155us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:55:04.363951 (+ 1291us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:55:04.363958 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:55:04.363995 (+ 37us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 02:55:04.364198 (+ 203us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:55:04.364209 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:55:04.365346 (+ 1137us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:55:04.365350 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:55:04.366504 (+ 1154us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:55:04.366514 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:55:04.367365 (+ 851us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:55:04.367369 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:55:04.367473 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:55:04.367510 (+ 37us) client_negotiation.cc:769] Sending connection context
0811 02:55:04.367643 (+ 133us) client_negotiation.cc:240] Negotiation successful
0811 02:55:04.367764 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":360,"mutex_wait_us":23,"spinlock_wait_cycles":7168,"thread_start_us":82,"threads_started":1}
I20260811 02:55:04.367936 31483 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:55:04.362097 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42896 (local address 127.12.138.195:35627)
0811 02:55:04.363455 (+ 1358us) server_negotiation.cc:206] Beginning negotiation
0811 02:55:04.363469 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0811 02:55:04.363492 (+ 23us) server_negotiation.cc:407] Connection header received
0811 02:55:04.363572 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:55:04.363577 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:55:04.363784 (+ 207us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:55:04.363943 (+ 159us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 02:55:04.364360 (+ 417us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:55:04.365198 (+ 838us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:55:04.366640 (+ 1442us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:55:04.367211 (+ 571us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:55:04.367373 (+ 162us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:55:04.367516 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0811 02:55:04.367685 (+ 169us) server_negotiation.cc:299] Negotiation successful
0811 02:55:04.367778 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1253,"thread_start_us":141,"threads_started":1}
I20260811 02:55:04.368767 31484 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:55:04.362632 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.138.194:42231 (local address 127.0.0.1:41246)
0811 02:55:04.363237 (+ 605us) negotiation.cc:107] Waiting for socket to connect
0811 02:55:04.363251 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 02:55:04.363344 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:55:04.363958 (+ 614us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:55:04.363961 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:55:04.363993 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 02:55:04.364204 (+ 211us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:55:04.364211 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:55:04.365814 (+ 1603us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:55:04.365818 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:55:04.367206 (+ 1388us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:55:04.367218 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:55:04.368305 (+ 1087us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:55:04.368308 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:55:04.368450 (+ 142us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:55:04.368474 (+ 24us) client_negotiation.cc:769] Sending connection context
0811 02:55:04.368539 (+ 65us) client_negotiation.cc:240] Negotiation successful
0811 02:55:04.368607 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":541,"mutex_wait_us":240,"spinlock_wait_cycles":5760,"thread_start_us":99,"threads_started":1}
I20260811 02:55:04.368934 31482 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:55:04.362122 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.138.193:41093 (local address 127.0.0.1:45050)
0811 02:55:04.363085 (+ 963us) negotiation.cc:107] Waiting for socket to connect
0811 02:55:04.363100 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 02:55:04.363197 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:55:04.364246 (+ 1049us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:55:04.364252 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:55:04.364274 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 02:55:04.364382 (+ 108us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:55:04.364388 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:55:04.365804 (+ 1416us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:55:04.365807 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:55:04.367206 (+ 1399us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:55:04.367218 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:55:04.368305 (+ 1087us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:55:04.368308 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:55:04.368500 (+ 192us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:55:04.368677 (+ 177us) client_negotiation.cc:769] Sending connection context
0811 02:55:04.368750 (+ 73us) client_negotiation.cc:240] Negotiation successful
0811 02:55:04.368817 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":899,"mutex_wait_us":146,"thread_start_us":110,"threads_started":1}
I20260811 02:55:04.369405 31485 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:55:04.362856 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41246 (local address 127.12.138.194:42231)
0811 02:55:04.363565 (+ 709us) server_negotiation.cc:206] Beginning negotiation
0811 02:55:04.363574 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 02:55:04.363591 (+ 17us) server_negotiation.cc:407] Connection header received
0811 02:55:04.363651 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:55:04.363654 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:55:04.363806 (+ 152us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:55:04.364064 (+ 258us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 02:55:04.364335 (+ 271us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:55:04.365675 (+ 1340us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:55:04.367355 (+ 1680us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:55:04.368160 (+ 805us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:55:04.368935 (+ 775us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:55:04.369067 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0811 02:55:04.369122 (+ 55us) server_negotiation.cc:299] Negotiation successful
0811 02:55:04.369254 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":625,"thread_start_us":139,"threads_started":1}
I20260811 02:55:04.369513 31487 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:55:04.363057 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45050 (local address 127.12.138.193:41093)
0811 02:55:04.363466 (+ 409us) server_negotiation.cc:206] Beginning negotiation
0811 02:55:04.363471 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 02:55:04.363492 (+ 21us) server_negotiation.cc:407] Connection header received
0811 02:55:04.363572 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:55:04.363577 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:55:04.363784 (+ 207us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:55:04.363943 (+ 159us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 02:55:04.364509 (+ 566us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:55:04.365647 (+ 1138us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:55:04.367355 (+ 1708us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:55:04.368153 (+ 798us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:55:04.368458 (+ 305us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:55:04.368598 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0811 02:55:04.369257 (+ 659us) server_negotiation.cc:299] Negotiation successful
0811 02:55:04.369378 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":317,"thread_start_us":120,"threads_started":1}
I20260811 02:55:04.371439 31389 tablet_service.cc:1467] Processing CreateTablet for tablet 343d72afa4354a1d8e587873e4f39120 (DEFAULT_TABLE table=test-table [id=848b08f4329043a3963890987035d6b7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 02:55:04.372521 31389 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 343d72afa4354a1d8e587873e4f39120. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 02:55:04.372246 31076 tablet_service.cc:1467] Processing CreateTablet for tablet 343d72afa4354a1d8e587873e4f39120 (DEFAULT_TABLE table=test-table [id=848b08f4329043a3963890987035d6b7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 02:55:04.372225 31228 tablet_service.cc:1467] Processing CreateTablet for tablet 343d72afa4354a1d8e587873e4f39120 (DEFAULT_TABLE table=test-table [id=848b08f4329043a3963890987035d6b7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 02:55:04.373339 31228 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 343d72afa4354a1d8e587873e4f39120. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 02:55:04.373339 31076 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 343d72afa4354a1d8e587873e4f39120. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 02:55:04.379431 31489 tablet_bootstrap.cc:492] T 343d72afa4354a1d8e587873e4f39120 P 924ad6fccb5f44c6a9da839c8b730081: Bootstrap starting.
I20260811 02:55:04.380934 31490 tablet_bootstrap.cc:492] T 343d72afa4354a1d8e587873e4f39120 P 93b8a8a73e364049893bd06522ab3d7c: Bootstrap starting.
I20260811 02:55:04.382696 31489 tablet_bootstrap.cc:654] T 343d72afa4354a1d8e587873e4f39120 P 924ad6fccb5f44c6a9da839c8b730081: Neither blocks nor log segments found. Creating new log.
I20260811 02:55:04.383648 31489 log.cc:824] T 343d72afa4354a1d8e587873e4f39120 P 924ad6fccb5f44c6a9da839c8b730081: Log is configured to *not* fsync() on all Append() calls
I20260811 02:55:04.384056 31490 tablet_bootstrap.cc:654] T 343d72afa4354a1d8e587873e4f39120 P 93b8a8a73e364049893bd06522ab3d7c: Neither blocks nor log segments found. Creating new log.
I20260811 02:55:04.384470 31491 tablet_bootstrap.cc:492] T 343d72afa4354a1d8e587873e4f39120 P b276426cf6d74110ac231a84ac3a3778: Bootstrap starting.
I20260811 02:55:04.384927 31490 log.cc:824] T 343d72afa4354a1d8e587873e4f39120 P 93b8a8a73e364049893bd06522ab3d7c: Log is configured to *not* fsync() on all Append() calls
I20260811 02:55:04.386011 31489 tablet_bootstrap.cc:492] T 343d72afa4354a1d8e587873e4f39120 P 924ad6fccb5f44c6a9da839c8b730081: No bootstrap required, opened a new log
I20260811 02:55:04.386210 31489 ts_tablet_manager.cc:1397] T 343d72afa4354a1d8e587873e4f39120 P 924ad6fccb5f44c6a9da839c8b730081: Time spent bootstrapping tablet: real 0.007s user 0.005s sys 0.000s
I20260811 02:55:04.387084 31490 tablet_bootstrap.cc:492] T 343d72afa4354a1d8e587873e4f39120 P 93b8a8a73e364049893bd06522ab3d7c: No bootstrap required, opened a new log
I20260811 02:55:04.387333 31491 tablet_bootstrap.cc:654] T 343d72afa4354a1d8e587873e4f39120 P b276426cf6d74110ac231a84ac3a3778: Neither blocks nor log segments found. Creating new log.
I20260811 02:55:04.387355 31490 ts_tablet_manager.cc:1397] T 343d72afa4354a1d8e587873e4f39120 P 93b8a8a73e364049893bd06522ab3d7c: Time spent bootstrapping tablet: real 0.007s user 0.005s sys 0.000s
I20260811 02:55:04.388243 31491 log.cc:824] T 343d72afa4354a1d8e587873e4f39120 P b276426cf6d74110ac231a84ac3a3778: Log is configured to *not* fsync() on all Append() calls
I20260811 02:55:04.391403 31491 tablet_bootstrap.cc:492] T 343d72afa4354a1d8e587873e4f39120 P b276426cf6d74110ac231a84ac3a3778: No bootstrap required, opened a new log
I20260811 02:55:04.391649 31491 ts_tablet_manager.cc:1397] T 343d72afa4354a1d8e587873e4f39120 P b276426cf6d74110ac231a84ac3a3778: Time spent bootstrapping tablet: real 0.007s user 0.004s sys 0.000s
I20260811 02:55:04.391680 31490 raft_consensus.cc:348] T 343d72afa4354a1d8e587873e4f39120 P 93b8a8a73e364049893bd06522ab3d7c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "924ad6fccb5f44c6a9da839c8b730081" member_type: VOTER last_known_addr { host: "127.12.138.195" port: 35627 } } peers { permanent_uuid: "b276426cf6d74110ac231a84ac3a3778" member_type: VOTER last_known_addr { host: "127.12.138.193" port: 41093 } } peers { permanent_uuid: "93b8a8a73e364049893bd06522ab3d7c" member_type: VOTER last_known_addr { host: "127.12.138.194" port: 42231 } }
I20260811 02:55:04.392089 31490 raft_consensus.cc:374] T 343d72afa4354a1d8e587873e4f39120 P 93b8a8a73e364049893bd06522ab3d7c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 02:55:04.392179 31490 raft_consensus.cc:729] T 343d72afa4354a1d8e587873e4f39120 P 93b8a8a73e364049893bd06522ab3d7c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 93b8a8a73e364049893bd06522ab3d7c, State: Initialized, Role: FOLLOWER
I20260811 02:55:04.392702 31490 consensus_queue.cc:260] T 343d72afa4354a1d8e587873e4f39120 P 93b8a8a73e364049893bd06522ab3d7c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "924ad6fccb5f44c6a9da839c8b730081" member_type: VOTER last_known_addr { host: "127.12.138.195" port: 35627 } } peers { permanent_uuid: "b276426cf6d74110ac231a84ac3a3778" member_type: VOTER last_known_addr { host: "127.12.138.193" port: 41093 } } peers { permanent_uuid: "93b8a8a73e364049893bd06522ab3d7c" member_type: VOTER last_known_addr { host: "127.12.138.194" port: 42231 } }
I20260811 02:55:04.392673 31489 raft_consensus.cc:348] T 343d72afa4354a1d8e587873e4f39120 P 924ad6fccb5f44c6a9da839c8b730081 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "924ad6fccb5f44c6a9da839c8b730081" member_type: VOTER last_known_addr { host: "127.12.138.195" port: 35627 } } peers { permanent_uuid: "b276426cf6d74110ac231a84ac3a3778" member_type: VOTER last_known_addr { host: "127.12.138.193" port: 41093 } } peers { permanent_uuid: "93b8a8a73e364049893bd06522ab3d7c" member_type: VOTER last_known_addr { host: "127.12.138.194" port: 42231 } }
I20260811 02:55:04.392937 31489 raft_consensus.cc:374] T 343d72afa4354a1d8e587873e4f39120 P 924ad6fccb5f44c6a9da839c8b730081 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 02:55:04.393002 31489 raft_consensus.cc:729] T 343d72afa4354a1d8e587873e4f39120 P 924ad6fccb5f44c6a9da839c8b730081 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 924ad6fccb5f44c6a9da839c8b730081, State: Initialized, Role: FOLLOWER
I20260811 02:55:04.393455 31489 consensus_queue.cc:260] T 343d72afa4354a1d8e587873e4f39120 P 924ad6fccb5f44c6a9da839c8b730081 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "924ad6fccb5f44c6a9da839c8b730081" member_type: VOTER last_known_addr { host: "127.12.138.195" port: 35627 } } peers { permanent_uuid: "b276426cf6d74110ac231a84ac3a3778" member_type: VOTER last_known_addr { host: "127.12.138.193" port: 41093 } } peers { permanent_uuid: "93b8a8a73e364049893bd06522ab3d7c" member_type: VOTER last_known_addr { host: "127.12.138.194" port: 42231 } }
I20260811 02:55:04.394024 31490 ts_tablet_manager.cc:1428] T 343d72afa4354a1d8e587873e4f39120 P 93b8a8a73e364049893bd06522ab3d7c: Time spent starting tablet: real 0.007s user 0.006s sys 0.000s
I20260811 02:55:04.394366 31489 ts_tablet_manager.cc:1428] T 343d72afa4354a1d8e587873e4f39120 P 924ad6fccb5f44c6a9da839c8b730081: Time spent starting tablet: real 0.008s user 0.006s sys 0.000s
I20260811 02:55:04.394379 31306 heartbeater.cc:503] Master 127.12.138.254:39391 was elected leader, sending a full tablet report...
I20260811 02:55:04.394544 31467 heartbeater.cc:503] Master 127.12.138.254:39391 was elected leader, sending a full tablet report...
I20260811 02:55:04.399389 31498 raft_consensus.cc:482] T 343d72afa4354a1d8e587873e4f39120 P 93b8a8a73e364049893bd06522ab3d7c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 02:55:04.399546 31498 raft_consensus.cc:504] T 343d72afa4354a1d8e587873e4f39120 P 93b8a8a73e364049893bd06522ab3d7c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "924ad6fccb5f44c6a9da839c8b730081" member_type: VOTER last_known_addr { host: "127.12.138.195" port: 35627 } } peers { permanent_uuid: "b276426cf6d74110ac231a84ac3a3778" member_type: VOTER last_known_addr { host: "127.12.138.193" port: 41093 } } peers { permanent_uuid: "93b8a8a73e364049893bd06522ab3d7c" member_type: VOTER last_known_addr { host: "127.12.138.194" port: 42231 } }
I20260811 02:55:04.399667 31491 raft_consensus.cc:348] T 343d72afa4354a1d8e587873e4f39120 P b276426cf6d74110ac231a84ac3a3778 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "924ad6fccb5f44c6a9da839c8b730081" member_type: VOTER last_known_addr { host: "127.12.138.195" port: 35627 } } peers { permanent_uuid: "b276426cf6d74110ac231a84ac3a3778" member_type: VOTER last_known_addr { host: "127.12.138.193" port: 41093 } } peers { permanent_uuid: "93b8a8a73e364049893bd06522ab3d7c" member_type: VOTER last_known_addr { host: "127.12.138.194" port: 42231 } }
I20260811 02:55:04.399952 31491 raft_consensus.cc:374] T 343d72afa4354a1d8e587873e4f39120 P b276426cf6d74110ac231a84ac3a3778 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 02:55:04.400028 31491 raft_consensus.cc:729] T 343d72afa4354a1d8e587873e4f39120 P b276426cf6d74110ac231a84ac3a3778 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b276426cf6d74110ac231a84ac3a3778, State: Initialized, Role: FOLLOWER
I20260811 02:55:04.400490 31491 consensus_queue.cc:260] T 343d72afa4354a1d8e587873e4f39120 P b276426cf6d74110ac231a84ac3a3778 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "924ad6fccb5f44c6a9da839c8b730081" member_type: VOTER last_known_addr { host: "127.12.138.195" port: 35627 } } peers { permanent_uuid: "b276426cf6d74110ac231a84ac3a3778" member_type: VOTER last_known_addr { host: "127.12.138.193" port: 41093 } } peers { permanent_uuid: "93b8a8a73e364049893bd06522ab3d7c" member_type: VOTER last_known_addr { host: "127.12.138.194" port: 42231 } }
I20260811 02:55:04.401168 31498 leader_election.cc:288] T 343d72afa4354a1d8e587873e4f39120 P 93b8a8a73e364049893bd06522ab3d7c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 924ad6fccb5f44c6a9da839c8b730081 (127.12.138.195:35627), b276426cf6d74110ac231a84ac3a3778 (127.12.138.193:41093)
I20260811 02:55:04.401226 31152 heartbeater.cc:503] Master 127.12.138.254:39391 was elected leader, sending a full tablet report...
I20260811 02:55:04.401855 31491 ts_tablet_manager.cc:1428] T 343d72afa4354a1d8e587873e4f39120 P b276426cf6d74110ac231a84ac3a3778: Time spent starting tablet: real 0.010s user 0.008s sys 0.000s
I20260811 02:55:04.406396 31487 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:55:04.401221 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.138.194:45533 (local address 127.12.138.193:41093)
0811 02:55:04.401337 (+ 116us) server_negotiation.cc:206] Beginning negotiation
0811 02:55:04.401341 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 02:55:04.402309 (+ 968us) server_negotiation.cc:407] Connection header received
0811 02:55:04.402491 (+ 182us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:55:04.402496 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:55:04.402547 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:55:04.402597 (+ 50us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 02:55:04.403276 (+ 679us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:55:04.404082 (+ 806us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:55:04.405312 (+ 1230us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:55:04.405806 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:55:04.405936 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:55:04.405987 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0811 02:55:04.406197 (+ 210us) server_negotiation.cc:299] Negotiation successful
0811 02:55:04.406267 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260811 02:55:04.406417 31308 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:55:04.401125 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.138.195:35627 (local address 127.12.138.194:48533)
0811 02:55:04.401431 (+ 306us) negotiation.cc:107] Waiting for socket to connect
0811 02:55:04.401448 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 02:55:04.401537 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:55:04.403097 (+ 1560us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:55:04.403099 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:55:04.403115 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 02:55:04.403231 (+ 116us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:55:04.403239 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:55:04.404203 (+ 964us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:55:04.404207 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:55:04.405217 (+ 1010us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:55:04.405232 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:55:04.405937 (+ 705us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:55:04.405940 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:55:04.406051 (+ 111us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:55:04.406094 (+ 43us) client_negotiation.cc:769] Sending connection context
0811 02:55:04.406194 (+ 100us) client_negotiation.cc:240] Negotiation successful
0811 02:55:04.406273 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":205,"mutex_wait_us":4}
I20260811 02:55:04.406414 31483 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:55:04.401709 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.138.194:48533 (local address 127.12.138.195:35627)
0811 02:55:04.402819 (+ 1110us) server_negotiation.cc:206] Beginning negotiation
0811 02:55:04.402823 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 02:55:04.402836 (+ 13us) server_negotiation.cc:407] Connection header received
0811 02:55:04.402879 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:55:04.402883 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:55:04.402921 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:55:04.402996 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 02:55:04.403366 (+ 370us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:55:04.404081 (+ 715us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:55:04.405366 (+ 1285us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:55:04.405809 (+ 443us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:55:04.405951 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:55:04.406007 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0811 02:55:04.406222 (+ 215us) server_negotiation.cc:299] Negotiation successful
0811 02:55:04.406295 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1032}
I20260811 02:55:04.406445 31500 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:55:04.401125 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.138.193:41093 (local address 127.12.138.194:45533)
0811 02:55:04.402226 (+ 1101us) negotiation.cc:107] Waiting for socket to connect
0811 02:55:04.402240 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 02:55:04.402372 (+ 132us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:55:04.402759 (+ 387us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:55:04.402762 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:55:04.402779 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 02:55:04.402917 (+ 138us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:55:04.402924 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:55:04.404240 (+ 1316us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:55:04.404243 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:55:04.405122 (+ 879us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:55:04.405131 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:55:04.405928 (+ 797us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:55:04.405931 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:55:04.406033 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:55:04.406087 (+ 54us) client_negotiation.cc:769] Sending connection context
0811 02:55:04.406182 (+ 95us) client_negotiation.cc:240] Negotiation successful
0811 02:55:04.406254 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":879,"mutex_wait_us":707,"thread_start_us":77,"threads_started":1}
I20260811 02:55:04.407066 31413 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "343d72afa4354a1d8e587873e4f39120" candidate_uuid: "93b8a8a73e364049893bd06522ab3d7c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "924ad6fccb5f44c6a9da839c8b730081" is_pre_election: true
I20260811 02:55:04.407307 31413 raft_consensus.cc:2454] T 343d72afa4354a1d8e587873e4f39120 P 924ad6fccb5f44c6a9da839c8b730081 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 93b8a8a73e364049893bd06522ab3d7c in term 0.
I20260811 02:55:04.407732 31175 leader_election.cc:302] T 343d72afa4354a1d8e587873e4f39120 P 93b8a8a73e364049893bd06522ab3d7c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 924ad6fccb5f44c6a9da839c8b730081, 93b8a8a73e364049893bd06522ab3d7c; no voters:
I20260811 02:55:04.408011 31498 raft_consensus.cc:2781] T 343d72afa4354a1d8e587873e4f39120 P 93b8a8a73e364049893bd06522ab3d7c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 02:55:04.408022 31100 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "343d72afa4354a1d8e587873e4f39120" candidate_uuid: "93b8a8a73e364049893bd06522ab3d7c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b276426cf6d74110ac231a84ac3a3778" is_pre_election: true
I20260811 02:55:04.408118 31498 raft_consensus.cc:482] T 343d72afa4354a1d8e587873e4f39120 P 93b8a8a73e364049893bd06522ab3d7c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 02:55:04.408176 31498 raft_consensus.cc:3037] T 343d72afa4354a1d8e587873e4f39120 P 93b8a8a73e364049893bd06522ab3d7c [term 0 FOLLOWER]: Advancing to term 1
I20260811 02:55:04.408278 31100 raft_consensus.cc:2454] T 343d72afa4354a1d8e587873e4f39120 P b276426cf6d74110ac231a84ac3a3778 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 93b8a8a73e364049893bd06522ab3d7c in term 0.
I20260811 02:55:04.409346 31498 raft_consensus.cc:504] T 343d72afa4354a1d8e587873e4f39120 P 93b8a8a73e364049893bd06522ab3d7c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "924ad6fccb5f44c6a9da839c8b730081" member_type: VOTER last_known_addr { host: "127.12.138.195" port: 35627 } } peers { permanent_uuid: "b276426cf6d74110ac231a84ac3a3778" member_type: VOTER last_known_addr { host: "127.12.138.193" port: 41093 } } peers { permanent_uuid: "93b8a8a73e364049893bd06522ab3d7c" member_type: VOTER last_known_addr { host: "127.12.138.194" port: 42231 } }
I20260811 02:55:04.409637 31498 leader_election.cc:288] T 343d72afa4354a1d8e587873e4f39120 P 93b8a8a73e364049893bd06522ab3d7c [CANDIDATE]: Term 1 election: Requested vote from peers 924ad6fccb5f44c6a9da839c8b730081 (127.12.138.195:35627), b276426cf6d74110ac231a84ac3a3778 (127.12.138.193:41093)
I20260811 02:55:04.409982 31413 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "343d72afa4354a1d8e587873e4f39120" candidate_uuid: "93b8a8a73e364049893bd06522ab3d7c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "924ad6fccb5f44c6a9da839c8b730081"
I20260811 02:55:04.409982 31100 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "343d72afa4354a1d8e587873e4f39120" candidate_uuid: "93b8a8a73e364049893bd06522ab3d7c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b276426cf6d74110ac231a84ac3a3778"
I20260811 02:55:04.410084 31413 raft_consensus.cc:3037] T 343d72afa4354a1d8e587873e4f39120 P 924ad6fccb5f44c6a9da839c8b730081 [term 0 FOLLOWER]: Advancing to term 1
I20260811 02:55:04.410128 31100 raft_consensus.cc:3037] T 343d72afa4354a1d8e587873e4f39120 P b276426cf6d74110ac231a84ac3a3778 [term 0 FOLLOWER]: Advancing to term 1
I20260811 02:55:04.411108 31413 raft_consensus.cc:2454] T 343d72afa4354a1d8e587873e4f39120 P 924ad6fccb5f44c6a9da839c8b730081 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 93b8a8a73e364049893bd06522ab3d7c in term 1.
I20260811 02:55:04.411394 31100 raft_consensus.cc:2454] T 343d72afa4354a1d8e587873e4f39120 P b276426cf6d74110ac231a84ac3a3778 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 93b8a8a73e364049893bd06522ab3d7c in term 1.
I20260811 02:55:04.411423 31175 leader_election.cc:302] T 343d72afa4354a1d8e587873e4f39120 P 93b8a8a73e364049893bd06522ab3d7c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 924ad6fccb5f44c6a9da839c8b730081, 93b8a8a73e364049893bd06522ab3d7c; no voters:
I20260811 02:55:04.411607 31498 raft_consensus.cc:2781] T 343d72afa4354a1d8e587873e4f39120 P 93b8a8a73e364049893bd06522ab3d7c [term 1 FOLLOWER]: Leader election won for term 1
I20260811 02:55:04.411720 31498 raft_consensus.cc:686] T 343d72afa4354a1d8e587873e4f39120 P 93b8a8a73e364049893bd06522ab3d7c [term 1 LEADER]: Becoming Leader. State: Replica: 93b8a8a73e364049893bd06522ab3d7c, State: Running, Role: LEADER
I20260811 02:55:04.412071 31498 consensus_queue.cc:237] T 343d72afa4354a1d8e587873e4f39120 P 93b8a8a73e364049893bd06522ab3d7c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "924ad6fccb5f44c6a9da839c8b730081" member_type: VOTER last_known_addr { host: "127.12.138.195" port: 35627 } } peers { permanent_uuid: "b276426cf6d74110ac231a84ac3a3778" member_type: VOTER last_known_addr { host: "127.12.138.193" port: 41093 } } peers { permanent_uuid: "93b8a8a73e364049893bd06522ab3d7c" member_type: VOTER last_known_addr { host: "127.12.138.194" port: 42231 } }
I20260811 02:55:04.415604 30935 catalog_manager.cc:5495] T 343d72afa4354a1d8e587873e4f39120 P 93b8a8a73e364049893bd06522ab3d7c reported cstate change: term changed from 0 to 1, leader changed from <none> to 93b8a8a73e364049893bd06522ab3d7c (127.12.138.194). New cstate: current_term: 1 leader_uuid: "93b8a8a73e364049893bd06522ab3d7c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "924ad6fccb5f44c6a9da839c8b730081" member_type: VOTER last_known_addr { host: "127.12.138.195" port: 35627 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b276426cf6d74110ac231a84ac3a3778" member_type: VOTER last_known_addr { host: "127.12.138.193" port: 41093 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "93b8a8a73e364049893bd06522ab3d7c" member_type: VOTER last_known_addr { host: "127.12.138.194" port: 42231 } health_report { overall_health: HEALTHY } } }
I20260811 02:55:04.434199 31485 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:55:04.430856 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41256 (local address 127.12.138.194:42231)
0811 02:55:04.430979 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0811 02:55:04.430983 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 02:55:04.430996 (+ 13us) server_negotiation.cc:407] Connection header received
0811 02:55:04.431136 (+ 140us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:55:04.431138 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:55:04.431191 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:55:04.431280 (+ 89us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 02:55:04.431520 (+ 240us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:55:04.432002 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:55:04.432826 (+ 824us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:55:04.433028 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:55:04.433100 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:55:04.433450 (+ 350us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 02:55:04.433854 (+ 404us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 02:55:04.433943 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0811 02:55:04.434026 (+ 83us) server_negotiation.cc:299] Negotiation successful
0811 02:55:04.434073 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260811 02:55:04.440470 31413 raft_consensus.cc:1261] T 343d72afa4354a1d8e587873e4f39120 P 924ad6fccb5f44c6a9da839c8b730081 [term 1 FOLLOWER]: Refusing update from remote peer 93b8a8a73e364049893bd06522ab3d7c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 02:55:04.440471 31100 raft_consensus.cc:1261] T 343d72afa4354a1d8e587873e4f39120 P b276426cf6d74110ac231a84ac3a3778 [term 1 FOLLOWER]: Refusing update from remote peer 93b8a8a73e364049893bd06522ab3d7c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 02:55:04.441299 31495 consensus_queue.cc:1035] T 343d72afa4354a1d8e587873e4f39120 P 93b8a8a73e364049893bd06522ab3d7c [LEADER]: Connected to new peer: Peer: permanent_uuid: "b276426cf6d74110ac231a84ac3a3778" member_type: VOTER last_known_addr { host: "127.12.138.193" port: 41093 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 02:55:04.441504 31498 consensus_queue.cc:1035] T 343d72afa4354a1d8e587873e4f39120 P 93b8a8a73e364049893bd06522ab3d7c [LEADER]: Connected to new peer: Peer: permanent_uuid: "924ad6fccb5f44c6a9da839c8b730081" member_type: VOTER last_known_addr { host: "127.12.138.195" port: 35627 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 02:55:04.450151 31505 mvcc.cc:205] Tried to move back new op lower bound from 7317163640579846144 to 7317163640475439104. Current Snapshot: MvccSnapshot[applied={T|T < 7317163640579846144}]
I20260811 02:55:04.458802 30933 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34990:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 02:55:04.459002 30933 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34990:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 02:55:04.461593 30933 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 4ce22d68cbc247349baf03912c7e6cb6: Sending DeleteTablet for 3 replicas of tablet 343d72afa4354a1d8e587873e4f39120
I20260811 02:55:04.462435 31076 tablet_service.cc:1514] Processing DeleteTablet for tablet 343d72afa4354a1d8e587873e4f39120 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 02:55:04 UTC) from {username='kudu', principal='kudu/127.12.138.254@KRBTEST.COM'} at 127.0.0.1:45050
I20260811 02:55:04.462460 31228 tablet_service.cc:1514] Processing DeleteTablet for tablet 343d72afa4354a1d8e587873e4f39120 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 02:55:04 UTC) from {username='kudu', principal='kudu/127.12.138.254@KRBTEST.COM'} at 127.0.0.1:41246
I20260811 02:55:04.462517 31389 tablet_service.cc:1514] Processing DeleteTablet for tablet 343d72afa4354a1d8e587873e4f39120 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 02:55:04 UTC) from {username='kudu', principal='kudu/127.12.138.254@KRBTEST.COM'} at 127.0.0.1:42896
I20260811 02:55:04.462957 31514 tablet_replica.cc:330] T 343d72afa4354a1d8e587873e4f39120 P 93b8a8a73e364049893bd06522ab3d7c: stopping tablet replica
I20260811 02:55:04.462960 31513 tablet_replica.cc:330] T 343d72afa4354a1d8e587873e4f39120 P b276426cf6d74110ac231a84ac3a3778: stopping tablet replica
I20260811 02:55:04.463271 31514 raft_consensus.cc:2229] T 343d72afa4354a1d8e587873e4f39120 P 93b8a8a73e364049893bd06522ab3d7c [term 1 LEADER]: Raft consensus shutting down.
I20260811 02:55:04.463271 31513 raft_consensus.cc:2229] T 343d72afa4354a1d8e587873e4f39120 P b276426cf6d74110ac231a84ac3a3778 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 02:55:04.463313 12843 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDawEwS/build/debug/bin/kudu with pid 31006
I20260811 02:55:04.478574 31514 raft_consensus.cc:2258] T 343d72afa4354a1d8e587873e4f39120 P 93b8a8a73e364049893bd06522ab3d7c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 02:55:04.480182 31514 ts_tablet_manager.cc:1905] T 343d72afa4354a1d8e587873e4f39120 P 93b8a8a73e364049893bd06522ab3d7c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 02:55:04.482677 31514 ts_tablet_manager.cc:1918] T 343d72afa4354a1d8e587873e4f39120 P 93b8a8a73e364049893bd06522ab3d7c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 02:55:04.482719 31515 tablet_replica.cc:330] T 343d72afa4354a1d8e587873e4f39120 P 924ad6fccb5f44c6a9da839c8b730081: stopping tablet replica
I20260811 02:55:04.482846 31514 log.cc:1196] T 343d72afa4354a1d8e587873e4f39120 P 93b8a8a73e364049893bd06522ab3d7c: Deleting WAL directory at /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786416707705494-12843-0/minicluster-data/ts-1/wal/wals/343d72afa4354a1d8e587873e4f39120
I20260811 02:55:04.483111 31515 raft_consensus.cc:2229] T 343d72afa4354a1d8e587873e4f39120 P 924ad6fccb5f44c6a9da839c8b730081 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 02:55:04.483270 31514 ts_tablet_manager.cc:1939] T 343d72afa4354a1d8e587873e4f39120 P 93b8a8a73e364049893bd06522ab3d7c: Deleting consensus metadata
I20260811 02:55:04.483503 31515 raft_consensus.cc:2258] T 343d72afa4354a1d8e587873e4f39120 P 924ad6fccb5f44c6a9da839c8b730081 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 02:55:04.484691 30918 catalog_manager.cc:4841] TS 93b8a8a73e364049893bd06522ab3d7c (127.12.138.194:42231): tablet 343d72afa4354a1d8e587873e4f39120 (table test-table [id=848b08f4329043a3963890987035d6b7]) successfully deleted
I20260811 02:55:04.485189 31515 ts_tablet_manager.cc:1905] T 343d72afa4354a1d8e587873e4f39120 P 924ad6fccb5f44c6a9da839c8b730081: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260811 02:55:04.485599 30919 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.12.138.193:41093 (error 108)
I20260811 02:55:04.485924 12843 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDawEwS/build/debug/bin/kudu with pid 31157
I20260811 02:55:04.488969 31515 ts_tablet_manager.cc:1918] T 343d72afa4354a1d8e587873e4f39120 P 924ad6fccb5f44c6a9da839c8b730081: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 02:55:04.489068 31515 log.cc:1196] T 343d72afa4354a1d8e587873e4f39120 P 924ad6fccb5f44c6a9da839c8b730081: Deleting WAL directory at /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786416707705494-12843-0/minicluster-data/ts-2/wal/wals/343d72afa4354a1d8e587873e4f39120
I20260811 02:55:04.489429 31515 ts_tablet_manager.cc:1939] T 343d72afa4354a1d8e587873e4f39120 P 924ad6fccb5f44c6a9da839c8b730081: Deleting consensus metadata
I20260811 02:55:04.490783 30917 catalog_manager.cc:4841] TS 924ad6fccb5f44c6a9da839c8b730081 (127.12.138.195:35627): tablet 343d72afa4354a1d8e587873e4f39120 (table test-table [id=848b08f4329043a3963890987035d6b7]) successfully deleted
I20260811 02:55:04.491364 31482 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:55:04.491056 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.138.193:41093 (local address 127.0.0.1:45054)
0811 02:55:04.491192 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0811 02:55:04.491269 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.138.193:41093: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
W20260811 02:55:04.491570 30919 catalog_manager.cc:4584] TS b276426cf6d74110ac231a84ac3a3778 (127.12.138.193:41093): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 343d72afa4354a1d8e587873e4f39120: Network error: Client connection negotiation failed: client connection to 127.12.138.193:41093: connect: Connection refused (error 111)
I20260811 02:55:04.496562 12843 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDawEwS/build/debug/bin/kudu with pid 31311
I20260811 02:55:04.507496 12843 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDawEwS/build/debug/bin/kudu with pid 30899
I20260811 02:55:24.516363 12843 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDawEwS/build/debug/bin/kudu
/tmp/dist-test-taskDawEwS/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786416707705494-12843-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786416707705494-12843-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786416707705494-12843-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786416707705494-12843-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.138.254:39391
--webserver_interface=127.12.138.254
--webserver_port=44325
--keytab_file=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786416707705494-12843-0/krb5kdc/kudu_127.12.138.254.keytab
--principal=kudu/127.12.138.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.138.212:42413
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.12.138.254:39391
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--tsk_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786416707705494-12843-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786416707705494-12843-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786416707705494-12843-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 02:55:24.637164 31517 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 02:55:24.637472 31517 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 02:55:24.637516 31517 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 02:55:24.641881 31517 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260811 02:55:24.641945 31517 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260811 02:55:24.641963 31517 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260811 02:55:24.641978 31517 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 02:55:24.642019 31517 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 02:55:24.642058 31517 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 02:55:24.642091 31517 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 02:55:24.642132 31517 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 02:55:24.647711 31517 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.138.212:42413
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786416707705494-12843-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786416707705494-12843-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_private_key_password_cmd=echo foo
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.12.138.254:39391
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786416707705494-12843-0/krb5kdc/kudu_127.12.138.254.keytab
--principal=kudu/127.12.138.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.138.254:39391
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786416707705494-12843-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.12.138.254
--webserver_port=44325
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31517
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786416707705494-12843-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 02:09:49 UTC on ccycloud.quasar-fsnqqc.root.comops.site
build id 4579570
I20260811 02:55:24.649024 31517 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 02:55:24.650226 31517 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 02:55:24.666054 31524 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:55:24.666121 31517 server_base.cc:1034] running on GCE node
W20260811 02:55:24.666054 31523 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:55:24.666056 31526 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:55:24.666996 31517 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 02:55:24.668248 31517 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 02:55:24.669417 31517 hybrid_clock.cc:648] HybridClock initialized: now 1786416924669379 us; error 52 us; skew 500 ppm
Aug 11 02:55:24 dist-test-slave-9gcw krb5kdc[30885](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416924, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.138.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 02:55:24.677125 31517 init.cc:377] Logged in from keytab as kudu/127.12.138.254@KRBTEST.COM (short username kudu)
I20260811 02:55:24.678175 31517 webserver.cc:460] Webserver started at http://127.12.138.254:44325/ using document root <none> and password file <none>
I20260811 02:55:24.678769 31517 fs_manager.cc:359] Metadata directory not provided
I20260811 02:55:24.678853 31517 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 02:55:24.682946 31517 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.001s
I20260811 02:55:24.685397 31535 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 02:55:24.686692 31517 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260811 02:55:24.686805 31517 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786416707705494-12843-0/minicluster-data/master-0/data,/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786416707705494-12843-0/minicluster-data/master-0/wal
uuid: "4ce22d68cbc247349baf03912c7e6cb6"
format_stamp: "Formatted at 2026-08-11 02:55:03 on dist-test-slave-9gcw"
server_key: "82b9b2173efd9043c9a0a661c2784363"
server_key_iv: "59f89185dfc326077347b28838f76d15"
server_key_version: "encryptionkey@0"
I20260811 02:55:24.687412 31517 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786416707705494-12843-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786416707705494-12843-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786416707705494-12843-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 02:55:24.704559 31517 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 02:55:24.707712 31517 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 02:55:24.707885 31517 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 02:55:24.715209 31517 rpc_server.cc:307] RPC server started. Bound to: 127.12.138.254:39391
I20260811 02:55:24.715546 31596 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.138.254:39391 every 8 connection(s)
I20260811 02:55:24.716496 31517 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786416707705494-12843-0/minicluster-data/master-0/data/info.pb
I20260811 02:55:24.721159 31597 sys_catalog.cc:263] Verifying existing consensus state
I20260811 02:55:24.723044 31597 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4ce22d68cbc247349baf03912c7e6cb6: Bootstrap starting.
I20260811 02:55:24.725446 12843 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDawEwS/build/debug/bin/kudu as pid 31517
I20260811 02:55:24.725943 12843 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDawEwS/build/debug/bin/kudu
/tmp/dist-test-taskDawEwS/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786416707705494-12843-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786416707705494-12843-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786416707705494-12843-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786416707705494-12843-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.138.193:41093
--local_ip_for_outbound_sockets=127.12.138.193
--tserver_master_addrs=127.12.138.254:39391
--webserver_port=41777
--webserver_interface=127.12.138.193
--keytab_file=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786416707705494-12843-0/krb5kdc/kudu_127.12.138.193.keytab
--principal=kudu/127.12.138.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.138.212:42413
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786416707705494-12843-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786416707705494-12843-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786416707705494-12843-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 02:55:24.742035 31597 log.cc:824] T 00000000000000000000000000000000 P 4ce22d68cbc247349baf03912c7e6cb6: Log is configured to *not* fsync() on all Append() calls
I20260811 02:55:24.754144 31597 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4ce22d68cbc247349baf03912c7e6cb6: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 02:55:24.754639 31597 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4ce22d68cbc247349baf03912c7e6cb6: Bootstrap complete.
I20260811 02:55:24.760390 31597 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4ce22d68cbc247349baf03912c7e6cb6 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4ce22d68cbc247349baf03912c7e6cb6" member_type: VOTER last_known_addr { host: "127.12.138.254" port: 39391 } }
I20260811 02:55:24.760936 31597 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4ce22d68cbc247349baf03912c7e6cb6 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4ce22d68cbc247349baf03912c7e6cb6, State: Initialized, Role: FOLLOWER
I20260811 02:55:24.761533 31597 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4ce22d68cbc247349baf03912c7e6cb6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4ce22d68cbc247349baf03912c7e6cb6" member_type: VOTER last_known_addr { host: "127.12.138.254" port: 39391 } }
I20260811 02:55:24.761710 31597 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4ce22d68cbc247349baf03912c7e6cb6 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 02:55:24.761790 31597 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4ce22d68cbc247349baf03912c7e6cb6 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 02:55:24.761881 31597 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4ce22d68cbc247349baf03912c7e6cb6 [term 1 FOLLOWER]: Advancing to term 2
I20260811 02:55:24.764565 31597 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4ce22d68cbc247349baf03912c7e6cb6 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4ce22d68cbc247349baf03912c7e6cb6" member_type: VOTER last_known_addr { host: "127.12.138.254" port: 39391 } }
I20260811 02:55:24.764966 31597 leader_election.cc:302] T 00000000000000000000000000000000 P 4ce22d68cbc247349baf03912c7e6cb6 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4ce22d68cbc247349baf03912c7e6cb6; no voters:
I20260811 02:55:24.765385 31597 leader_election.cc:288] T 00000000000000000000000000000000 P 4ce22d68cbc247349baf03912c7e6cb6 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260811 02:55:24.765553 31601 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4ce22d68cbc247349baf03912c7e6cb6 [term 2 FOLLOWER]: Leader election won for term 2
I20260811 02:55:24.765841 31601 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4ce22d68cbc247349baf03912c7e6cb6 [term 2 LEADER]: Becoming Leader. State: Replica: 4ce22d68cbc247349baf03912c7e6cb6, State: Running, Role: LEADER
I20260811 02:55:24.766170 31601 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4ce22d68cbc247349baf03912c7e6cb6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4ce22d68cbc247349baf03912c7e6cb6" member_type: VOTER last_known_addr { host: "127.12.138.254" port: 39391 } }
I20260811 02:55:24.766770 31597 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4ce22d68cbc247349baf03912c7e6cb6 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 02:55:24.767670 31604 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4ce22d68cbc247349baf03912c7e6cb6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4ce22d68cbc247349baf03912c7e6cb6. Latest consensus state: current_term: 2 leader_uuid: "4ce22d68cbc247349baf03912c7e6cb6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4ce22d68cbc247349baf03912c7e6cb6" member_type: VOTER last_known_addr { host: "127.12.138.254" port: 39391 } } }
I20260811 02:55:24.767812 31604 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4ce22d68cbc247349baf03912c7e6cb6 [sys.catalog]: This master's current role is: LEADER
I20260811 02:55:24.768115 31602 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4ce22d68cbc247349baf03912c7e6cb6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "4ce22d68cbc247349baf03912c7e6cb6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4ce22d68cbc247349baf03912c7e6cb6" member_type: VOTER last_known_addr { host: "127.12.138.254" port: 39391 } } }
I20260811 02:55:24.768213 31602 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4ce22d68cbc247349baf03912c7e6cb6 [sys.catalog]: This master's current role is: LEADER
I20260811 02:55:24.768507 31614 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 02:55:24.773947 31614 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 02:55:24.774641 31614 catalog_manager.cc:1259] Loaded cluster ID: ef117953dc34499cbebe471f50cf1592
I20260811 02:55:24.774701 31614 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 02:55:24.776527 31614 catalog_manager.cc:1504] Loading token signing keys...
I20260811 02:55:24.777244 31614 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 4ce22d68cbc247349baf03912c7e6cb6: Loaded TSK: 0
I20260811 02:55:24.789955 31614 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4ce22d68cbc247349baf03912c7e6cb6: Generated new TSK 1
I20260811 02:55:24.790272 31614 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260811 02:55:24.865250 31599 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 02:55:24.865530 31599 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 02:55:24.865574 31599 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 02:55:24.870095 31599 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 02:55:24.870162 31599 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 02:55:24.870253 31599 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.138.193
I20260811 02:55:24.875694 31599 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.138.212:42413
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786416707705494-12843-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786416707705494-12843-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786416707705494-12843-0/krb5kdc/kudu_127.12.138.193.keytab
--principal=kudu/127.12.138.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.138.193:41093
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786416707705494-12843-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.12.138.193
--webserver_port=41777
--tserver_master_addrs=127.12.138.254:39391
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31599
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.138.193
--log_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786416707705494-12843-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 02:09:49 UTC on ccycloud.quasar-fsnqqc.root.comops.site
build id 4579570
I20260811 02:55:24.876797 31599 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 02:55:24.877929 31599 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 02:55:24.888562 31628 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:55:24.888564 31629 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:55:24.889106 31631 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:55:24.889392 31599 server_base.cc:1034] running on GCE node
I20260811 02:55:24.889856 31599 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 02:55:24.890621 31599 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 02:55:24.891808 31599 hybrid_clock.cc:648] HybridClock initialized: now 1786416924891781 us; error 44 us; skew 500 ppm
Aug 11 02:55:24 dist-test-slave-9gcw krb5kdc[30885](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416924, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.138.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 02:55:24.894928 31599 init.cc:377] Logged in from keytab as kudu/127.12.138.193@KRBTEST.COM (short username kudu)
I20260811 02:55:24.896049 31599 webserver.cc:460] Webserver started at http://127.12.138.193:41777/ using document root <none> and password file <none>
I20260811 02:55:24.896665 31599 fs_manager.cc:359] Metadata directory not provided
I20260811 02:55:24.896785 31599 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 02:55:24.901067 31599 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.000s
I20260811 02:55:24.903450 31640 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 02:55:24.904825 31599 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260811 02:55:24.904961 31599 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786416707705494-12843-0/minicluster-data/ts-0/data,/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786416707705494-12843-0/minicluster-data/ts-0/wal
uuid: "b276426cf6d74110ac231a84ac3a3778"
format_stamp: "Formatted at 2026-08-11 02:55:03 on dist-test-slave-9gcw"
server_key: "4705960a372f5dbbafe86ae8d0016d6a"
server_key_iv: "c017c9d055ebf8a2f24ec71073419281"
server_key_version: "encryptionkey@0"
I20260811 02:55:24.905606 31599 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786416707705494-12843-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786416707705494-12843-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786416707705494-12843-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 02:55:24.947825 31599 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 02:55:24.951678 31599 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 02:55:24.951915 31599 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 02:55:24.952613 31599 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 02:55:24.953929 31650 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260811 02:55:24.958607 31599 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260811 02:55:24.958674 31599 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.005s user 0.001s sys 0.000s
I20260811 02:55:24.958751 31599 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260811 02:55:24.960520 31599 ts_tablet_manager.cc:610] Registered 1 tablets
I20260811 02:55:24.960572 31599 ts_tablet_manager.cc:589] Time spent register tablets: real 0.002s user 0.000s sys 0.000s
I20260811 02:55:24.961014 31650 tablet_bootstrap.cc:492] T 343d72afa4354a1d8e587873e4f39120 P b276426cf6d74110ac231a84ac3a3778: Bootstrap starting.
I20260811 02:55:24.973644 31599 rpc_server.cc:307] RPC server started. Bound to: 127.12.138.193:41093
I20260811 02:55:24.973735 31772 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.138.193:41093 every 8 connection(s)
I20260811 02:55:24.974774 31599 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786416707705494-12843-0/minicluster-data/ts-0/data/info.pb
I20260811 02:55:24.975628 12843 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDawEwS/build/debug/bin/kudu as pid 31599
I20260811 02:55:24.976168 12843 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDawEwS/build/debug/bin/kudu
/tmp/dist-test-taskDawEwS/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786416707705494-12843-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786416707705494-12843-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786416707705494-12843-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786416707705494-12843-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.138.194:42231
--local_ip_for_outbound_sockets=127.12.138.194
--tserver_master_addrs=127.12.138.254:39391
--webserver_port=33595
--webserver_interface=127.12.138.194
--keytab_file=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786416707705494-12843-0/krb5kdc/kudu_127.12.138.194.keytab
--principal=kudu/127.12.138.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.138.212:42413
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786416707705494-12843-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786416707705494-12843-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786416707705494-12843-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 02:55:24.980582 31650 log.cc:824] T 343d72afa4354a1d8e587873e4f39120 P b276426cf6d74110ac231a84ac3a3778: Log is configured to *not* fsync() on all Append() calls
I20260811 02:55:24.988607 31650 tablet_bootstrap.cc:492] T 343d72afa4354a1d8e587873e4f39120 P b276426cf6d74110ac231a84ac3a3778: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 02:55:24.989048 31650 tablet_bootstrap.cc:492] T 343d72afa4354a1d8e587873e4f39120 P b276426cf6d74110ac231a84ac3a3778: Bootstrap complete.
Aug 11 02:55:24 dist-test-slave-9gcw krb5kdc[30885](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416924, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.138.193@KRBTEST.COM for kudu/127.12.138.254@KRBTEST.COM
I20260811 02:55:24.990607 31650 ts_tablet_manager.cc:1397] T 343d72afa4354a1d8e587873e4f39120 P b276426cf6d74110ac231a84ac3a3778: Time spent bootstrapping tablet: real 0.030s user 0.028s sys 0.000s
I20260811 02:55:24.993577 31650 raft_consensus.cc:348] T 343d72afa4354a1d8e587873e4f39120 P b276426cf6d74110ac231a84ac3a3778 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "924ad6fccb5f44c6a9da839c8b730081" member_type: VOTER last_known_addr { host: "127.12.138.195" port: 35627 } } peers { permanent_uuid: "b276426cf6d74110ac231a84ac3a3778" member_type: VOTER last_known_addr { host: "127.12.138.193" port: 41093 } } peers { permanent_uuid: "93b8a8a73e364049893bd06522ab3d7c" member_type: VOTER last_known_addr { host: "127.12.138.194" port: 42231 } }
I20260811 02:55:24.994186 31650 raft_consensus.cc:729] T 343d72afa4354a1d8e587873e4f39120 P b276426cf6d74110ac231a84ac3a3778 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: b276426cf6d74110ac231a84ac3a3778, State: Initialized, Role: FOLLOWER
I20260811 02:55:24.994699 31650 consensus_queue.cc:260] T 343d72afa4354a1d8e587873e4f39120 P b276426cf6d74110ac231a84ac3a3778 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "924ad6fccb5f44c6a9da839c8b730081" member_type: VOTER last_known_addr { host: "127.12.138.195" port: 35627 } } peers { permanent_uuid: "b276426cf6d74110ac231a84ac3a3778" member_type: VOTER last_known_addr { host: "127.12.138.193" port: 41093 } } peers { permanent_uuid: "93b8a8a73e364049893bd06522ab3d7c" member_type: VOTER last_known_addr { host: "127.12.138.194" port: 42231 } }
I20260811 02:55:24.995793 31650 ts_tablet_manager.cc:1428] T 343d72afa4354a1d8e587873e4f39120 P b276426cf6d74110ac231a84ac3a3778: Time spent starting tablet: real 0.005s user 0.004s sys 0.000s
I20260811 02:55:24.997066 31775 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:55:24.977134 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.138.193:46897 (local address 127.12.138.254:39391)
0811 02:55:24.977736 (+ 602us) server_negotiation.cc:206] Beginning negotiation
0811 02:55:24.977759 (+ 23us) server_negotiation.cc:399] Waiting for connection header
0811 02:55:24.978336 (+ 577us) server_negotiation.cc:407] Connection header received
0811 02:55:24.980192 (+ 1856us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:55:24.980209 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:55:24.980637 (+ 428us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:55:24.981060 (+ 423us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:55:24.982700 (+ 1640us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:55:24.984244 (+ 1544us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:55:24.986325 (+ 2081us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:55:24.987017 (+ 692us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:55:24.987189 (+ 172us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:55:24.991012 (+ 3823us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:55:24.991023 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:55:24.991041 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:55:24.991086 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:55:24.993442 (+ 2356us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:55:24.994108 (+ 666us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:55:24.994114 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:55:24.994119 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:55:24.994389 (+ 270us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:55:24.994820 (+ 431us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:55:24.994824 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:55:24.994826 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:55:24.995330 (+ 504us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:55:24.995491 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0811 02:55:24.996162 (+ 671us) server_negotiation.cc:299] Negotiation successful
0811 02:55:24.996409 (+ 247us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":427,"thread_start_us":149,"threads_started":1}
I20260811 02:55:24.997260 31776 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:55:24.977214 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.138.254:39391 (local address 127.12.138.193:46897)
0811 02:55:24.978113 (+ 899us) negotiation.cc:107] Waiting for socket to connect
0811 02:55:24.978176 (+ 63us) client_negotiation.cc:174] Beginning negotiation
0811 02:55:24.979623 (+ 1447us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:55:24.981161 (+ 1538us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:55:24.981175 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:55:24.981751 (+ 576us) client_negotiation.cc:189] Negotiated authn=SASL
0811 02:55:24.982462 (+ 711us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:55:24.982487 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:55:24.984454 (+ 1967us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:55:24.984460 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:55:24.986160 (+ 1700us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:55:24.986177 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:55:24.987192 (+ 1015us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:55:24.987199 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:55:24.987475 (+ 276us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:55:24.988270 (+ 795us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 02:55:24.988297 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0811 02:55:24.990590 (+ 2293us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 02:55:24.993593 (+ 3003us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:55:24.993602 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:55:24.993624 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:55:24.993971 (+ 347us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:55:24.994509 (+ 538us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:55:24.994512 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:55:24.994514 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:55:24.994691 (+ 177us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:55:24.995460 (+ 769us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 02:55:24.995466 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 02:55:24.995770 (+ 304us) client_negotiation.cc:769] Sending connection context
0811 02:55:24.995967 (+ 197us) client_negotiation.cc:240] Negotiation successful
0811 02:55:24.996340 (+ 373us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":681,"thread_start_us":151,"threads_started":1}
I20260811 02:55:24.999781 31773 heartbeater.cc:347] Connected to a master server at 127.12.138.254:39391
I20260811 02:55:25.000036 31773 heartbeater.cc:464] Registering TS with master...
I20260811 02:55:25.000571 31773 heartbeater.cc:511] Master 127.12.138.254:39391 requested a full tablet report, sending...
I20260811 02:55:25.003103 31553 ts_manager.cc:194] Registered new tserver with Master: b276426cf6d74110ac231a84ac3a3778 (127.12.138.193:41093)
I20260811 02:55:25.006779 31553 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.12.138.193@KRBTEST.COM'} at 127.12.138.193:46897
I20260811 02:55:25.009799 31773 heartbeater.cc:503] Master 127.12.138.254:39391 was elected leader, sending a full tablet report...
Aug 11 02:55:25 dist-test-slave-9gcw krb5kdc[30885](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416924, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.138.254@KRBTEST.COM for kudu/127.12.138.193@KRBTEST.COM
I20260811 02:55:25.017728 31781 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:55:25.006140 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.138.193:41093 (local address 127.0.0.1:43004)
0811 02:55:25.007079 (+ 939us) negotiation.cc:107] Waiting for socket to connect
0811 02:55:25.007124 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0811 02:55:25.007271 (+ 147us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:55:25.007813 (+ 542us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:55:25.007820 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:55:25.008198 (+ 378us) client_negotiation.cc:189] Negotiated authn=SASL
0811 02:55:25.008311 (+ 113us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:55:25.008318 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:55:25.009433 (+ 1115us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:55:25.009437 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:55:25.010392 (+ 955us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:55:25.010405 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:55:25.010991 (+ 586us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:55:25.010995 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:55:25.011112 (+ 117us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:55:25.011801 (+ 689us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 02:55:25.011814 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0811 02:55:25.013505 (+ 1691us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 02:55:25.015904 (+ 2399us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:55:25.015913 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:55:25.015917 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:55:25.016153 (+ 236us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:55:25.016502 (+ 349us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:55:25.016504 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:55:25.016505 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:55:25.016586 (+ 81us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:55:25.017117 (+ 531us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 02:55:25.017122 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 02:55:25.017301 (+ 179us) client_negotiation.cc:769] Sending connection context
0811 02:55:25.017424 (+ 123us) client_negotiation.cc:240] Negotiation successful
0811 02:55:25.017589 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":686,"thread_start_us":132,"threads_started":1}
I20260811 02:55:25.018004 31782 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:55:25.006272 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43004 (local address 127.12.138.193:41093)
0811 02:55:25.006717 (+ 445us) server_negotiation.cc:206] Beginning negotiation
0811 02:55:25.006730 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0811 02:55:25.007234 (+ 504us) server_negotiation.cc:407] Connection header received
0811 02:55:25.007429 (+ 195us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:55:25.007435 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:55:25.007640 (+ 205us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:55:25.007789 (+ 149us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:55:25.008443 (+ 654us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:55:25.009315 (+ 872us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:55:25.010523 (+ 1208us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:55:25.010867 (+ 344us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:55:25.010985 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:55:25.013728 (+ 2743us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:55:25.013739 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:55:25.013751 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:55:25.013779 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:55:25.015785 (+ 2006us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:55:25.016285 (+ 500us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:55:25.016290 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:55:25.016292 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:55:25.016403 (+ 111us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:55:25.016680 (+ 277us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:55:25.016683 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:55:25.016685 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:55:25.016993 (+ 308us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:55:25.017169 (+ 176us) server_negotiation.cc:1035] Waiting for connection context
0811 02:55:25.017544 (+ 375us) server_negotiation.cc:299] Negotiation successful
0811 02:55:25.017778 (+ 234us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":340,"thread_start_us":125,"threads_started":1}
I20260811 02:55:25.018952 31695 tablet_service.cc:1514] Processing DeleteTablet for tablet 343d72afa4354a1d8e587873e4f39120 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 02:55:04 UTC) from {username='kudu', principal='kudu/127.12.138.254@KRBTEST.COM'} at 127.0.0.1:43004
I20260811 02:55:25.019847 31785 tablet_replica.cc:330] stopping tablet replica
I20260811 02:55:25.020160 31785 raft_consensus.cc:2229] T 343d72afa4354a1d8e587873e4f39120 P b276426cf6d74110ac231a84ac3a3778 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 02:55:25.020421 31785 raft_consensus.cc:2258] T 343d72afa4354a1d8e587873e4f39120 P b276426cf6d74110ac231a84ac3a3778 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 02:55:25.021571 31785 ts_tablet_manager.cc:1905] T 343d72afa4354a1d8e587873e4f39120 P b276426cf6d74110ac231a84ac3a3778: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 02:55:25.025364 31785 ts_tablet_manager.cc:1918] T 343d72afa4354a1d8e587873e4f39120 P b276426cf6d74110ac231a84ac3a3778: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 02:55:25.025513 31785 log.cc:1196] T 343d72afa4354a1d8e587873e4f39120 P b276426cf6d74110ac231a84ac3a3778: Deleting WAL directory at /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786416707705494-12843-0/minicluster-data/ts-0/wal/wals/343d72afa4354a1d8e587873e4f39120
I20260811 02:55:25.025803 31785 ts_tablet_manager.cc:1939] T 343d72afa4354a1d8e587873e4f39120 P b276426cf6d74110ac231a84ac3a3778: Deleting consensus metadata
I20260811 02:55:25.027793 31538 catalog_manager.cc:4841] TS b276426cf6d74110ac231a84ac3a3778 (127.12.138.193:41093): tablet 343d72afa4354a1d8e587873e4f39120 (table test-table [id=848b08f4329043a3963890987035d6b7]) successfully deleted
W20260811 02:55:25.107442 31777 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 02:55:25.107829 31777 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 02:55:25.107915 31777 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 02:55:25.112694 31777 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 02:55:25.112805 31777 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 02:55:25.112915 31777 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.138.194
I20260811 02:55:25.118916 31777 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.138.212:42413
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786416707705494-12843-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786416707705494-12843-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786416707705494-12843-0/krb5kdc/kudu_127.12.138.194.keytab
--principal=kudu/127.12.138.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.138.194:42231
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786416707705494-12843-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.12.138.194
--webserver_port=33595
--tserver_master_addrs=127.12.138.254:39391
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31777
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.138.194
--log_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786416707705494-12843-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 02:09:49 UTC on ccycloud.quasar-fsnqqc.root.comops.site
build id 4579570
I20260811 02:55:25.120406 31777 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 02:55:25.121928 31777 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 02:55:25.133049 31792 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:55:25.133025 31791 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:55:25.133109 31777 server_base.cc:1034] running on GCE node
W20260811 02:55:25.133028 31794 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:55:25.133873 31777 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 02:55:25.134577 31777 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 02:55:25.135807 31777 hybrid_clock.cc:648] HybridClock initialized: now 1786416925135765 us; error 51 us; skew 500 ppm
Aug 11 02:55:25 dist-test-slave-9gcw krb5kdc[30885](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416925, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.138.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 02:55:25.138839 31777 init.cc:377] Logged in from keytab as kudu/127.12.138.194@KRBTEST.COM (short username kudu)
I20260811 02:55:25.139715 31777 webserver.cc:460] Webserver started at http://127.12.138.194:33595/ using document root <none> and password file <none>
I20260811 02:55:25.140256 31777 fs_manager.cc:359] Metadata directory not provided
I20260811 02:55:25.140332 31777 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 02:55:25.144536 31777 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.000s
I20260811 02:55:25.147063 31802 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 02:55:25.148510 31777 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260811 02:55:25.148684 31777 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786416707705494-12843-0/minicluster-data/ts-1/data,/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786416707705494-12843-0/minicluster-data/ts-1/wal
uuid: "93b8a8a73e364049893bd06522ab3d7c"
format_stamp: "Formatted at 2026-08-11 02:55:04 on dist-test-slave-9gcw"
server_key: "adb77886557a128a3658d3768cfeb27d"
server_key_iv: "73633ad8d01bc30fbc2dc442c7d12812"
server_key_version: "encryptionkey@0"
I20260811 02:55:25.149392 31777 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786416707705494-12843-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786416707705494-12843-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786416707705494-12843-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 02:55:25.168905 31777 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 02:55:25.172179 31777 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 02:55:25.172415 31777 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 02:55:25.173031 31777 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 02:55:25.173926 31777 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 02:55:25.174001 31777 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 02:55:25.174077 31777 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 02:55:25.174129 31777 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 02:55:25.183303 31777 rpc_server.cc:307] RPC server started. Bound to: 127.12.138.194:42231
I20260811 02:55:25.183329 31931 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.138.194:42231 every 8 connection(s)
I20260811 02:55:25.184365 31777 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786416707705494-12843-0/minicluster-data/ts-1/data/info.pb
I20260811 02:55:25.186831 12843 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDawEwS/build/debug/bin/kudu as pid 31777
I20260811 02:55:25.187388 12843 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDawEwS/build/debug/bin/kudu
/tmp/dist-test-taskDawEwS/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786416707705494-12843-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786416707705494-12843-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786416707705494-12843-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786416707705494-12843-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.138.195:35627
--local_ip_for_outbound_sockets=127.12.138.195
--tserver_master_addrs=127.12.138.254:39391
--webserver_port=37877
--webserver_interface=127.12.138.195
--keytab_file=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786416707705494-12843-0/krb5kdc/kudu_127.12.138.195.keytab
--principal=kudu/127.12.138.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.138.212:42413
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786416707705494-12843-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786416707705494-12843-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786416707705494-12843-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 11 02:55:25 dist-test-slave-9gcw krb5kdc[30885](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416925, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.138.194@KRBTEST.COM for kudu/127.12.138.254@KRBTEST.COM
I20260811 02:55:25.201283 31775 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:55:25.186252 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.138.194:46021 (local address 127.12.138.254:39391)
0811 02:55:25.186397 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0811 02:55:25.186400 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 02:55:25.187582 (+ 1182us) server_negotiation.cc:407] Connection header received
0811 02:55:25.189034 (+ 1452us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:55:25.189037 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:55:25.189089 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:55:25.189263 (+ 174us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:55:25.190528 (+ 1265us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:55:25.191175 (+ 647us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:55:25.192971 (+ 1796us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:55:25.193287 (+ 316us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:55:25.193414 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:55:25.196643 (+ 3229us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:55:25.196646 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:55:25.196648 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:55:25.196674 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:55:25.198771 (+ 2097us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:55:25.199329 (+ 558us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:55:25.199333 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:55:25.199337 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:55:25.199445 (+ 108us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:55:25.199805 (+ 360us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:55:25.199810 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:55:25.199815 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:55:25.200134 (+ 319us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:55:25.200253 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0811 02:55:25.200781 (+ 528us) server_negotiation.cc:299] Negotiation successful
0811 02:55:25.200955 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260811 02:55:25.201972 31934 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:55:25.186601 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.138.254:39391 (local address 127.12.138.194:46021)
0811 02:55:25.187407 (+ 806us) negotiation.cc:107] Waiting for socket to connect
0811 02:55:25.187470 (+ 63us) client_negotiation.cc:174] Beginning negotiation
0811 02:55:25.188777 (+ 1307us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:55:25.189456 (+ 679us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:55:25.189469 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:55:25.189840 (+ 371us) client_negotiation.cc:189] Negotiated authn=SASL
0811 02:55:25.190331 (+ 491us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:55:25.190345 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:55:25.191320 (+ 975us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:55:25.191326 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:55:25.192839 (+ 1513us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:55:25.192854 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:55:25.193467 (+ 613us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:55:25.193475 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:55:25.193725 (+ 250us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:55:25.194390 (+ 665us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 02:55:25.194417 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0811 02:55:25.196427 (+ 2010us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 02:55:25.198910 (+ 2483us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:55:25.198915 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:55:25.198929 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:55:25.199208 (+ 279us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:55:25.199557 (+ 349us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:55:25.199559 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:55:25.199560 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:55:25.199703 (+ 143us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:55:25.200255 (+ 552us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 02:55:25.200260 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 02:55:25.200518 (+ 258us) client_negotiation.cc:769] Sending connection context
0811 02:55:25.200687 (+ 169us) client_negotiation.cc:240] Negotiation successful
0811 02:55:25.200976 (+ 289us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":547,"thread_start_us":107,"threads_started":1}
I20260811 02:55:25.203222 31932 heartbeater.cc:347] Connected to a master server at 127.12.138.254:39391
I20260811 02:55:25.203492 31932 heartbeater.cc:464] Registering TS with master...
I20260811 02:55:25.204033 31932 heartbeater.cc:511] Master 127.12.138.254:39391 requested a full tablet report, sending...
I20260811 02:55:25.205121 31553 ts_manager.cc:194] Registered new tserver with Master: 93b8a8a73e364049893bd06522ab3d7c (127.12.138.194:42231)
I20260811 02:55:25.205770 31553 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.12.138.194@KRBTEST.COM'} at 127.12.138.194:46021
W20260811 02:55:25.309278 31937 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 02:55:25.309643 31937 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 02:55:25.309711 31937 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 02:55:25.313884 31937 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 02:55:25.313941 31937 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 02:55:25.314034 31937 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.138.195
I20260811 02:55:25.319402 31937 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.138.212:42413
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786416707705494-12843-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786416707705494-12843-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786416707705494-12843-0/krb5kdc/kudu_127.12.138.195.keytab
--principal=kudu/127.12.138.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.138.195:35627
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786416707705494-12843-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.12.138.195
--webserver_port=37877
--tserver_master_addrs=127.12.138.254:39391
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31937
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.138.195
--log_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786416707705494-12843-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 02:09:49 UTC on ccycloud.quasar-fsnqqc.root.comops.site
build id 4579570
I20260811 02:55:25.320484 31937 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 02:55:25.321695 31937 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 02:55:25.331667 31942 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:55:25.331676 31945 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:55:25.331676 31943 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:55:25.332058 31937 server_base.cc:1034] running on GCE node
I20260811 02:55:25.332471 31937 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 02:55:25.333097 31937 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 02:55:25.334314 31937 hybrid_clock.cc:648] HybridClock initialized: now 1786416925334282 us; error 47 us; skew 500 ppm
Aug 11 02:55:25 dist-test-slave-9gcw krb5kdc[30885](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416925, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.138.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 02:55:25.337122 31937 init.cc:377] Logged in from keytab as kudu/127.12.138.195@KRBTEST.COM (short username kudu)
I20260811 02:55:25.338135 31937 webserver.cc:460] Webserver started at http://127.12.138.195:37877/ using document root <none> and password file <none>
I20260811 02:55:25.338721 31937 fs_manager.cc:359] Metadata directory not provided
I20260811 02:55:25.338838 31937 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 02:55:25.342886 31937 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260811 02:55:25.345008 31952 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 02:55:25.346340 31937 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260811 02:55:25.346494 31937 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786416707705494-12843-0/minicluster-data/ts-2/data,/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786416707705494-12843-0/minicluster-data/ts-2/wal
uuid: "924ad6fccb5f44c6a9da839c8b730081"
format_stamp: "Formatted at 2026-08-11 02:55:04 on dist-test-slave-9gcw"
server_key: "a63d3bf5d003f94c5a55c3705422df2e"
server_key_iv: "436cefcc8a36da9bf0941d370ca47a77"
server_key_version: "encryptionkey@0"
I20260811 02:55:25.347083 31937 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786416707705494-12843-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786416707705494-12843-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786416707705494-12843-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 02:55:25.362573 31937 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 02:55:25.365664 31937 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 02:55:25.365880 31937 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 02:55:25.366537 31937 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 02:55:25.367406 31937 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 02:55:25.367482 31937 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 02:55:25.367550 31937 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 02:55:25.367600 31937 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 02:55:25.377012 31937 rpc_server.cc:307] RPC server started. Bound to: 127.12.138.195:35627
I20260811 02:55:25.377041 32081 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.138.195:35627 every 8 connection(s)
I20260811 02:55:25.378146 31937 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786416707705494-12843-0/minicluster-data/ts-2/data/info.pb
I20260811 02:55:25.386955 12843 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDawEwS/build/debug/bin/kudu as pid 31937
Aug 11 02:55:25 dist-test-slave-9gcw krb5kdc[30885](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416925, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.138.195@KRBTEST.COM for kudu/127.12.138.254@KRBTEST.COM
I20260811 02:55:25.393134 31775 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:55:25.380048 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.138.195:41009 (local address 127.12.138.254:39391)
0811 02:55:25.380197 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0811 02:55:25.380201 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 02:55:25.381376 (+ 1175us) server_negotiation.cc:407] Connection header received
0811 02:55:25.382342 (+ 966us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:55:25.382345 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:55:25.382405 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:55:25.382495 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:55:25.383781 (+ 1286us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:55:25.384325 (+ 544us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:55:25.385604 (+ 1279us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:55:25.385845 (+ 241us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:55:25.385967 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:55:25.388968 (+ 3001us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:55:25.388970 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:55:25.388972 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:55:25.388992 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:55:25.390644 (+ 1652us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:55:25.391197 (+ 553us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:55:25.391201 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:55:25.391203 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:55:25.391302 (+ 99us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:55:25.391703 (+ 401us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:55:25.391713 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:55:25.391714 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:55:25.391965 (+ 251us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:55:25.392070 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0811 02:55:25.392737 (+ 667us) server_negotiation.cc:299] Negotiation successful
0811 02:55:25.392890 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58,"spinlock_wait_cycles":7424}
I20260811 02:55:25.393996 32086 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:55:25.380475 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.138.254:39391 (local address 127.12.138.195:41009)
0811 02:55:25.381182 (+ 707us) negotiation.cc:107] Waiting for socket to connect
0811 02:55:25.381263 (+ 81us) client_negotiation.cc:174] Beginning negotiation
0811 02:55:25.382117 (+ 854us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:55:25.382705 (+ 588us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:55:25.382712 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:55:25.383087 (+ 375us) client_negotiation.cc:189] Negotiated authn=SASL
0811 02:55:25.383619 (+ 532us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:55:25.383632 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:55:25.384477 (+ 845us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:55:25.384485 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:55:25.385472 (+ 987us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:55:25.385482 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:55:25.386008 (+ 526us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:55:25.386013 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:55:25.386191 (+ 178us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:55:25.386870 (+ 679us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 02:55:25.386888 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0811 02:55:25.388785 (+ 1897us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 02:55:25.390759 (+ 1974us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:55:25.390770 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:55:25.390783 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:55:25.391097 (+ 314us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:55:25.391390 (+ 293us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:55:25.391413 (+ 23us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:55:25.391418 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:55:25.391600 (+ 182us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:55:25.392096 (+ 496us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 02:55:25.392106 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 02:55:25.392444 (+ 338us) client_negotiation.cc:769] Sending connection context
0811 02:55:25.392650 (+ 206us) client_negotiation.cc:240] Negotiation successful
0811 02:55:25.392929 (+ 279us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":494,"thread_start_us":117,"threads_started":1}
I20260811 02:55:25.395308 32084 heartbeater.cc:347] Connected to a master server at 127.12.138.254:39391
I20260811 02:55:25.395606 32084 heartbeater.cc:464] Registering TS with master...
I20260811 02:55:25.396257 32084 heartbeater.cc:511] Master 127.12.138.254:39391 requested a full tablet report, sending...
I20260811 02:55:25.397289 32088 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:55:25.387744 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37472 (local address 127.12.138.254:39391)
0811 02:55:25.388045 (+ 301us) server_negotiation.cc:206] Beginning negotiation
0811 02:55:25.388050 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 02:55:25.388218 (+ 168us) server_negotiation.cc:407] Connection header received
0811 02:55:25.388384 (+ 166us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:55:25.388387 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:55:25.388439 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:55:25.388524 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:55:25.389452 (+ 928us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:55:25.390158 (+ 706us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:55:25.391449 (+ 1291us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:55:25.391821 (+ 372us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:55:25.391898 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:55:25.393534 (+ 1636us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:55:25.393537 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:55:25.393540 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:55:25.393563 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:55:25.395196 (+ 1633us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:55:25.395700 (+ 504us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:55:25.395704 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:55:25.395705 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:55:25.395799 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:55:25.396091 (+ 292us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:55:25.396094 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:55:25.396096 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:55:25.396310 (+ 214us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:55:25.396519 (+ 209us) server_negotiation.cc:1035] Waiting for connection context
0811 02:55:25.396928 (+ 409us) server_negotiation.cc:299] Negotiation successful
0811 02:55:25.397068 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":212,"thread_start_us":102,"threads_started":1}
I20260811 02:55:25.397596 31553 ts_manager.cc:194] Registered new tserver with Master: 924ad6fccb5f44c6a9da839c8b730081 (127.12.138.195:35627)
I20260811 02:55:25.398262 31553 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.12.138.195@KRBTEST.COM'} at 127.12.138.195:41009
I20260811 02:55:25.399320 12843 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 02:55:25.403635 32088 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:55:25.399974 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37484 (local address 127.12.138.254:39391)
0811 02:55:25.400101 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0811 02:55:25.400106 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 02:55:25.400334 (+ 228us) server_negotiation.cc:407] Connection header received
0811 02:55:25.400522 (+ 188us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:55:25.400525 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:55:25.400611 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:55:25.400720 (+ 109us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 02:55:25.400964 (+ 244us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:55:25.401540 (+ 576us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:55:25.402424 (+ 884us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:55:25.402659 (+ 235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:55:25.402772 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:55:25.403109 (+ 337us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 02:55:25.403231 (+ 122us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_AUTHENTICATION_TOKEN: Not authorized: token expired
0811 02:55:25.403498 (+ 267us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:37484: token expired
Metrics: {"server-negotiator.queue_time_us":47}
W20260811 02:55:25.403692 32088 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:37484: token expired
W20260811 02:55:25.404146 31472 master_proxy_rpc.cc:203] Re-attempting CreateTable request to leader Master (127.12.138.254:39391)
I20260811 02:55:25.411736 32088 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:55:25.404539 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37494 (local address 127.12.138.254:39391)
0811 02:55:25.404651 (+ 112us) server_negotiation.cc:206] Beginning negotiation
0811 02:55:25.404654 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 02:55:25.404686 (+ 32us) server_negotiation.cc:407] Connection header received
0811 02:55:25.404797 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:55:25.404799 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:55:25.404832 (+ 33us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:55:25.404893 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:55:25.405661 (+ 768us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:55:25.406129 (+ 468us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:55:25.407019 (+ 890us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:55:25.407215 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:55:25.407293 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:55:25.408725 (+ 1432us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:55:25.408728 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:55:25.408729 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:55:25.408751 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:55:25.410314 (+ 1563us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:55:25.410702 (+ 388us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:55:25.410704 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:55:25.410706 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:55:25.410794 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:55:25.411025 (+ 231us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:55:25.411027 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:55:25.411028 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:55:25.411185 (+ 157us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:55:25.411283 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0811 02:55:25.411492 (+ 209us) server_negotiation.cc:299] Negotiation successful
0811 02:55:25.411622 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260811 02:55:25.414250 31552 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37494:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 02:55:25.417522 31552 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 02:55:25.425544 31695 tablet_service.cc:1467] Processing CreateTablet for tablet 9cf2eb209cfa4faeae176a09a287c435 (DEFAULT_TABLE table=test-table [id=e3de53e34c3b43c7aaae8893d9fb65af]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 02:55:25.426280 31695 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9cf2eb209cfa4faeae176a09a287c435. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 02:55:25.429411 31650 tablet_bootstrap.cc:492] T 9cf2eb209cfa4faeae176a09a287c435 P b276426cf6d74110ac231a84ac3a3778: Bootstrap starting.
I20260811 02:55:25.430397 31650 tablet_bootstrap.cc:654] T 9cf2eb209cfa4faeae176a09a287c435 P b276426cf6d74110ac231a84ac3a3778: Neither blocks nor log segments found. Creating new log.
I20260811 02:55:25.430513 31781 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:55:25.424282 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.138.194:42231 (local address 127.0.0.1:43270)
0811 02:55:25.424408 (+ 126us) negotiation.cc:107] Waiting for socket to connect
0811 02:55:25.424424 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 02:55:25.424502 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:55:25.425463 (+ 961us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:55:25.425466 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:55:25.425485 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 02:55:25.425601 (+ 116us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:55:25.425608 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:55:25.427199 (+ 1591us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:55:25.427205 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:55:25.428786 (+ 1581us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:55:25.428798 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:55:25.429915 (+ 1117us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:55:25.429919 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:55:25.430126 (+ 207us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:55:25.430157 (+ 31us) client_negotiation.cc:769] Sending connection context
0811 02:55:25.430227 (+ 70us) client_negotiation.cc:240] Negotiation successful
0811 02:55:25.430324 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":34,"mutex_wait_us":407}
I20260811 02:55:25.430583 32094 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:55:25.424480 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43270 (local address 127.12.138.194:42231)
0811 02:55:25.424874 (+ 394us) server_negotiation.cc:206] Beginning negotiation
0811 02:55:25.424887 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0811 02:55:25.424915 (+ 28us) server_negotiation.cc:407] Connection header received
0811 02:55:25.424996 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:55:25.425002 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:55:25.425268 (+ 266us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:55:25.425774 (+ 506us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 02:55:25.425980 (+ 206us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:55:25.427062 (+ 1082us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:55:25.428936 (+ 1874us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:55:25.429769 (+ 833us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:55:25.430141 (+ 372us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:55:25.430283 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0811 02:55:25.430337 (+ 54us) server_negotiation.cc:299] Negotiation successful
0811 02:55:25.430437 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":267,"thread_start_us":161,"threads_started":1}
I20260811 02:55:25.430895 32096 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:55:25.425196 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34374 (local address 127.12.138.195:35627)
0811 02:55:25.425624 (+ 428us) server_negotiation.cc:206] Beginning negotiation
0811 02:55:25.425641 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0811 02:55:25.425659 (+ 18us) server_negotiation.cc:407] Connection header received
0811 02:55:25.425721 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:55:25.425724 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:55:25.425939 (+ 215us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:55:25.426093 (+ 154us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 02:55:25.426311 (+ 218us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:55:25.427087 (+ 776us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:55:25.428740 (+ 1653us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:55:25.429769 (+ 1029us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:55:25.429910 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:55:25.430060 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0811 02:55:25.430605 (+ 545us) server_negotiation.cc:299] Negotiation successful
0811 02:55:25.430722 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":350,"thread_start_us":120,"threads_started":1}
I20260811 02:55:25.430527 32095 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:55:25.424994 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.138.195:35627 (local address 127.0.0.1:34374)
0811 02:55:25.425331 (+ 337us) negotiation.cc:107] Waiting for socket to connect
0811 02:55:25.425347 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 02:55:25.425433 (+ 86us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:55:25.426073 (+ 640us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:55:25.426075 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:55:25.426093 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 02:55:25.426205 (+ 112us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:55:25.426214 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:55:25.427238 (+ 1024us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:55:25.427241 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:55:25.428606 (+ 1365us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:55:25.428616 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:55:25.429886 (+ 1270us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:55:25.429890 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:55:25.430067 (+ 177us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:55:25.430143 (+ 76us) client_negotiation.cc:769] Sending connection context
0811 02:55:25.430225 (+ 82us) client_negotiation.cc:240] Negotiation successful
0811 02:55:25.430348 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":260,"mutex_wait_us":93,"thread_start_us":95,"threads_started":1}
I20260811 02:55:25.432217 31650 tablet_bootstrap.cc:492] T 9cf2eb209cfa4faeae176a09a287c435 P b276426cf6d74110ac231a84ac3a3778: No bootstrap required, opened a new log
I20260811 02:55:25.432312 31650 ts_tablet_manager.cc:1397] T 9cf2eb209cfa4faeae176a09a287c435 P b276426cf6d74110ac231a84ac3a3778: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260811 02:55:25.432739 31650 raft_consensus.cc:348] T 9cf2eb209cfa4faeae176a09a287c435 P b276426cf6d74110ac231a84ac3a3778 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "93b8a8a73e364049893bd06522ab3d7c" member_type: VOTER last_known_addr { host: "127.12.138.194" port: 42231 } } peers { permanent_uuid: "924ad6fccb5f44c6a9da839c8b730081" member_type: VOTER last_known_addr { host: "127.12.138.195" port: 35627 } } peers { permanent_uuid: "b276426cf6d74110ac231a84ac3a3778" member_type: VOTER last_known_addr { host: "127.12.138.193" port: 41093 } }
I20260811 02:55:25.432876 31650 raft_consensus.cc:374] T 9cf2eb209cfa4faeae176a09a287c435 P b276426cf6d74110ac231a84ac3a3778 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 02:55:25.432914 31650 raft_consensus.cc:729] T 9cf2eb209cfa4faeae176a09a287c435 P b276426cf6d74110ac231a84ac3a3778 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b276426cf6d74110ac231a84ac3a3778, State: Initialized, Role: FOLLOWER
I20260811 02:55:25.433041 31650 consensus_queue.cc:260] T 9cf2eb209cfa4faeae176a09a287c435 P b276426cf6d74110ac231a84ac3a3778 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "93b8a8a73e364049893bd06522ab3d7c" member_type: VOTER last_known_addr { host: "127.12.138.194" port: 42231 } } peers { permanent_uuid: "924ad6fccb5f44c6a9da839c8b730081" member_type: VOTER last_known_addr { host: "127.12.138.195" port: 35627 } } peers { permanent_uuid: "b276426cf6d74110ac231a84ac3a3778" member_type: VOTER last_known_addr { host: "127.12.138.193" port: 41093 } }
I20260811 02:55:25.433351 31650 ts_tablet_manager.cc:1428] T 9cf2eb209cfa4faeae176a09a287c435 P b276426cf6d74110ac231a84ac3a3778: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260811 02:55:25.434589 31854 tablet_service.cc:1467] Processing CreateTablet for tablet 9cf2eb209cfa4faeae176a09a287c435 (DEFAULT_TABLE table=test-table [id=e3de53e34c3b43c7aaae8893d9fb65af]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 02:55:25.435839 31854 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9cf2eb209cfa4faeae176a09a287c435. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 02:55:25.435567 32003 tablet_service.cc:1467] Processing CreateTablet for tablet 9cf2eb209cfa4faeae176a09a287c435 (DEFAULT_TABLE table=test-table [id=e3de53e34c3b43c7aaae8893d9fb65af]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 02:55:25.436609 32003 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9cf2eb209cfa4faeae176a09a287c435. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 02:55:25.444569 32098 tablet_bootstrap.cc:492] T 9cf2eb209cfa4faeae176a09a287c435 P 924ad6fccb5f44c6a9da839c8b730081: Bootstrap starting.
I20260811 02:55:25.445489 32099 tablet_bootstrap.cc:492] T 9cf2eb209cfa4faeae176a09a287c435 P 93b8a8a73e364049893bd06522ab3d7c: Bootstrap starting.
I20260811 02:55:25.447170 32098 tablet_bootstrap.cc:654] T 9cf2eb209cfa4faeae176a09a287c435 P 924ad6fccb5f44c6a9da839c8b730081: Neither blocks nor log segments found. Creating new log.
I20260811 02:55:25.447711 32099 tablet_bootstrap.cc:654] T 9cf2eb209cfa4faeae176a09a287c435 P 93b8a8a73e364049893bd06522ab3d7c: Neither blocks nor log segments found. Creating new log.
I20260811 02:55:25.447983 32098 log.cc:824] T 9cf2eb209cfa4faeae176a09a287c435 P 924ad6fccb5f44c6a9da839c8b730081: Log is configured to *not* fsync() on all Append() calls
I20260811 02:55:25.448468 32099 log.cc:824] T 9cf2eb209cfa4faeae176a09a287c435 P 93b8a8a73e364049893bd06522ab3d7c: Log is configured to *not* fsync() on all Append() calls
I20260811 02:55:25.450316 32098 tablet_bootstrap.cc:492] T 9cf2eb209cfa4faeae176a09a287c435 P 924ad6fccb5f44c6a9da839c8b730081: No bootstrap required, opened a new log
I20260811 02:55:25.450547 32098 ts_tablet_manager.cc:1397] T 9cf2eb209cfa4faeae176a09a287c435 P 924ad6fccb5f44c6a9da839c8b730081: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260811 02:55:25.450695 32099 tablet_bootstrap.cc:492] T 9cf2eb209cfa4faeae176a09a287c435 P 93b8a8a73e364049893bd06522ab3d7c: No bootstrap required, opened a new log
I20260811 02:55:25.450912 32099 ts_tablet_manager.cc:1397] T 9cf2eb209cfa4faeae176a09a287c435 P 93b8a8a73e364049893bd06522ab3d7c: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260811 02:55:25.453617 32098 raft_consensus.cc:348] T 9cf2eb209cfa4faeae176a09a287c435 P 924ad6fccb5f44c6a9da839c8b730081 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "93b8a8a73e364049893bd06522ab3d7c" member_type: VOTER last_known_addr { host: "127.12.138.194" port: 42231 } } peers { permanent_uuid: "924ad6fccb5f44c6a9da839c8b730081" member_type: VOTER last_known_addr { host: "127.12.138.195" port: 35627 } } peers { permanent_uuid: "b276426cf6d74110ac231a84ac3a3778" member_type: VOTER last_known_addr { host: "127.12.138.193" port: 41093 } }
I20260811 02:55:25.453855 32098 raft_consensus.cc:374] T 9cf2eb209cfa4faeae176a09a287c435 P 924ad6fccb5f44c6a9da839c8b730081 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 02:55:25.453936 32098 raft_consensus.cc:729] T 9cf2eb209cfa4faeae176a09a287c435 P 924ad6fccb5f44c6a9da839c8b730081 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 924ad6fccb5f44c6a9da839c8b730081, State: Initialized, Role: FOLLOWER
I20260811 02:55:25.454115 32099 raft_consensus.cc:348] T 9cf2eb209cfa4faeae176a09a287c435 P 93b8a8a73e364049893bd06522ab3d7c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "93b8a8a73e364049893bd06522ab3d7c" member_type: VOTER last_known_addr { host: "127.12.138.194" port: 42231 } } peers { permanent_uuid: "924ad6fccb5f44c6a9da839c8b730081" member_type: VOTER last_known_addr { host: "127.12.138.195" port: 35627 } } peers { permanent_uuid: "b276426cf6d74110ac231a84ac3a3778" member_type: VOTER last_known_addr { host: "127.12.138.193" port: 41093 } }
I20260811 02:55:25.454404 32099 raft_consensus.cc:374] T 9cf2eb209cfa4faeae176a09a287c435 P 93b8a8a73e364049893bd06522ab3d7c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 02:55:25.454471 32099 raft_consensus.cc:729] T 9cf2eb209cfa4faeae176a09a287c435 P 93b8a8a73e364049893bd06522ab3d7c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 93b8a8a73e364049893bd06522ab3d7c, State: Initialized, Role: FOLLOWER
I20260811 02:55:25.454391 32098 consensus_queue.cc:260] T 9cf2eb209cfa4faeae176a09a287c435 P 924ad6fccb5f44c6a9da839c8b730081 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "93b8a8a73e364049893bd06522ab3d7c" member_type: VOTER last_known_addr { host: "127.12.138.194" port: 42231 } } peers { permanent_uuid: "924ad6fccb5f44c6a9da839c8b730081" member_type: VOTER last_known_addr { host: "127.12.138.195" port: 35627 } } peers { permanent_uuid: "b276426cf6d74110ac231a84ac3a3778" member_type: VOTER last_known_addr { host: "127.12.138.193" port: 41093 } }
I20260811 02:55:25.454923 32099 consensus_queue.cc:260] T 9cf2eb209cfa4faeae176a09a287c435 P 93b8a8a73e364049893bd06522ab3d7c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "93b8a8a73e364049893bd06522ab3d7c" member_type: VOTER last_known_addr { host: "127.12.138.194" port: 42231 } } peers { permanent_uuid: "924ad6fccb5f44c6a9da839c8b730081" member_type: VOTER last_known_addr { host: "127.12.138.195" port: 35627 } } peers { permanent_uuid: "b276426cf6d74110ac231a84ac3a3778" member_type: VOTER last_known_addr { host: "127.12.138.193" port: 41093 } }
I20260811 02:55:25.455168 32084 heartbeater.cc:503] Master 127.12.138.254:39391 was elected leader, sending a full tablet report...
I20260811 02:55:25.455528 32098 ts_tablet_manager.cc:1428] T 9cf2eb209cfa4faeae176a09a287c435 P 924ad6fccb5f44c6a9da839c8b730081: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260811 02:55:25.455937 32099 ts_tablet_manager.cc:1428] T 9cf2eb209cfa4faeae176a09a287c435 P 93b8a8a73e364049893bd06522ab3d7c: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260811 02:55:25.456118 31932 heartbeater.cc:503] Master 127.12.138.254:39391 was elected leader, sending a full tablet report...
W20260811 02:55:25.476460 31774 tablet.cc:2369] T 9cf2eb209cfa4faeae176a09a287c435 P b276426cf6d74110ac231a84ac3a3778: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 02:55:25.586891 31779 raft_consensus.cc:482] T 9cf2eb209cfa4faeae176a09a287c435 P b276426cf6d74110ac231a84ac3a3778 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 02:55:25.587129 31779 raft_consensus.cc:504] T 9cf2eb209cfa4faeae176a09a287c435 P b276426cf6d74110ac231a84ac3a3778 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "93b8a8a73e364049893bd06522ab3d7c" member_type: VOTER last_known_addr { host: "127.12.138.194" port: 42231 } } peers { permanent_uuid: "924ad6fccb5f44c6a9da839c8b730081" member_type: VOTER last_known_addr { host: "127.12.138.195" port: 35627 } } peers { permanent_uuid: "b276426cf6d74110ac231a84ac3a3778" member_type: VOTER last_known_addr { host: "127.12.138.193" port: 41093 } }
I20260811 02:55:25.588265 31779 leader_election.cc:288] T 9cf2eb209cfa4faeae176a09a287c435 P b276426cf6d74110ac231a84ac3a3778 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 93b8a8a73e364049893bd06522ab3d7c (127.12.138.194:42231), 924ad6fccb5f44c6a9da839c8b730081 (127.12.138.195:35627)
I20260811 02:55:25.593842 32096 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:55:25.588694 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.138.193:52241 (local address 127.12.138.195:35627)
0811 02:55:25.588860 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0811 02:55:25.588864 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 02:55:25.588993 (+ 129us) server_negotiation.cc:407] Connection header received
0811 02:55:25.589177 (+ 184us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:55:25.589180 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:55:25.589232 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:55:25.589318 (+ 86us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 02:55:25.589738 (+ 420us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:55:25.590565 (+ 827us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:55:25.592482 (+ 1917us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:55:25.593112 (+ 630us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:55:25.593268 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:55:25.593325 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0811 02:55:25.593625 (+ 300us) server_negotiation.cc:299] Negotiation successful
0811 02:55:25.593700 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260811 02:55:25.593847 32094 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:55:25.588609 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.138.193:53183 (local address 127.12.138.194:42231)
0811 02:55:25.588744 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0811 02:55:25.588749 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 02:55:25.588979 (+ 230us) server_negotiation.cc:407] Connection header received
0811 02:55:25.589176 (+ 197us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:55:25.589179 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:55:25.589234 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:55:25.589321 (+ 87us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 02:55:25.589738 (+ 417us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:55:25.590547 (+ 809us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:55:25.592482 (+ 1935us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:55:25.593113 (+ 631us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:55:25.593268 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:55:25.593325 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0811 02:55:25.593625 (+ 300us) server_negotiation.cc:299] Negotiation successful
0811 02:55:25.593700 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260811 02:55:25.593899 32104 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:55:25.588512 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.138.194:42231 (local address 127.12.138.193:53183)
0811 02:55:25.588847 (+ 335us) negotiation.cc:107] Waiting for socket to connect
0811 02:55:25.588863 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 02:55:25.589002 (+ 139us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:55:25.589353 (+ 351us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:55:25.589357 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:55:25.589396 (+ 39us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 02:55:25.589615 (+ 219us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:55:25.589623 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:55:25.590673 (+ 1050us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:55:25.590676 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:55:25.592335 (+ 1659us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:55:25.592348 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:55:25.593282 (+ 934us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:55:25.593285 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:55:25.593420 (+ 135us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:55:25.593499 (+ 79us) client_negotiation.cc:769] Sending connection context
0811 02:55:25.593586 (+ 87us) client_negotiation.cc:240] Negotiation successful
0811 02:55:25.593717 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":263,"mutex_wait_us":266,"spinlock_wait_cycles":7680,"thread_start_us":82,"threads_started":1}
I20260811 02:55:25.594421 32105 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:55:25.588535 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.138.195:35627 (local address 127.12.138.193:52241)
0811 02:55:25.588900 (+ 365us) negotiation.cc:107] Waiting for socket to connect
0811 02:55:25.588914 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 02:55:25.589002 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:55:25.589353 (+ 351us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:55:25.589357 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:55:25.589396 (+ 39us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 02:55:25.589615 (+ 219us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:55:25.589623 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:55:25.590673 (+ 1050us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:55:25.590676 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:55:25.592335 (+ 1659us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:55:25.592348 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:55:25.593275 (+ 927us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:55:25.593280 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:55:25.593424 (+ 144us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:55:25.593504 (+ 80us) client_negotiation.cc:769] Sending connection context
0811 02:55:25.593586 (+ 82us) client_negotiation.cc:240] Negotiation successful
0811 02:55:25.593711 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":299,"mutex_wait_us":334,"spinlock_wait_cycles":34560,"thread_start_us":103,"threads_started":1}
I20260811 02:55:25.594638 31876 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9cf2eb209cfa4faeae176a09a287c435" candidate_uuid: "b276426cf6d74110ac231a84ac3a3778" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "93b8a8a73e364049893bd06522ab3d7c" is_pre_election: true
I20260811 02:55:25.595000 31876 raft_consensus.cc:2454] T 9cf2eb209cfa4faeae176a09a287c435 P 93b8a8a73e364049893bd06522ab3d7c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b276426cf6d74110ac231a84ac3a3778 in term 0.
I20260811 02:55:25.595147 32026 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9cf2eb209cfa4faeae176a09a287c435" candidate_uuid: "b276426cf6d74110ac231a84ac3a3778" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "924ad6fccb5f44c6a9da839c8b730081" is_pre_election: true
I20260811 02:55:25.595433 32026 raft_consensus.cc:2454] T 9cf2eb209cfa4faeae176a09a287c435 P 924ad6fccb5f44c6a9da839c8b730081 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b276426cf6d74110ac231a84ac3a3778 in term 0.
I20260811 02:55:25.595939 31643 leader_election.cc:302] T 9cf2eb209cfa4faeae176a09a287c435 P b276426cf6d74110ac231a84ac3a3778 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 924ad6fccb5f44c6a9da839c8b730081, 93b8a8a73e364049893bd06522ab3d7c, b276426cf6d74110ac231a84ac3a3778; no voters:
I20260811 02:55:25.596254 31779 raft_consensus.cc:2781] T 9cf2eb209cfa4faeae176a09a287c435 P b276426cf6d74110ac231a84ac3a3778 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 02:55:25.596340 31779 raft_consensus.cc:482] T 9cf2eb209cfa4faeae176a09a287c435 P b276426cf6d74110ac231a84ac3a3778 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 02:55:25.596374 31779 raft_consensus.cc:3037] T 9cf2eb209cfa4faeae176a09a287c435 P b276426cf6d74110ac231a84ac3a3778 [term 0 FOLLOWER]: Advancing to term 1
I20260811 02:55:25.597446 31779 raft_consensus.cc:504] T 9cf2eb209cfa4faeae176a09a287c435 P b276426cf6d74110ac231a84ac3a3778 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "93b8a8a73e364049893bd06522ab3d7c" member_type: VOTER last_known_addr { host: "127.12.138.194" port: 42231 } } peers { permanent_uuid: "924ad6fccb5f44c6a9da839c8b730081" member_type: VOTER last_known_addr { host: "127.12.138.195" port: 35627 } } peers { permanent_uuid: "b276426cf6d74110ac231a84ac3a3778" member_type: VOTER last_known_addr { host: "127.12.138.193" port: 41093 } }
I20260811 02:55:25.597784 31779 leader_election.cc:288] T 9cf2eb209cfa4faeae176a09a287c435 P b276426cf6d74110ac231a84ac3a3778 [CANDIDATE]: Term 1 election: Requested vote from peers 93b8a8a73e364049893bd06522ab3d7c (127.12.138.194:42231), 924ad6fccb5f44c6a9da839c8b730081 (127.12.138.195:35627)
I20260811 02:55:25.598131 31876 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9cf2eb209cfa4faeae176a09a287c435" candidate_uuid: "b276426cf6d74110ac231a84ac3a3778" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "93b8a8a73e364049893bd06522ab3d7c"
I20260811 02:55:25.598151 32026 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9cf2eb209cfa4faeae176a09a287c435" candidate_uuid: "b276426cf6d74110ac231a84ac3a3778" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "924ad6fccb5f44c6a9da839c8b730081"
I20260811 02:55:25.598270 31876 raft_consensus.cc:3037] T 9cf2eb209cfa4faeae176a09a287c435 P 93b8a8a73e364049893bd06522ab3d7c [term 0 FOLLOWER]: Advancing to term 1
I20260811 02:55:25.598322 32026 raft_consensus.cc:3037] T 9cf2eb209cfa4faeae176a09a287c435 P 924ad6fccb5f44c6a9da839c8b730081 [term 0 FOLLOWER]: Advancing to term 1
I20260811 02:55:25.599362 31876 raft_consensus.cc:2454] T 9cf2eb209cfa4faeae176a09a287c435 P 93b8a8a73e364049893bd06522ab3d7c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b276426cf6d74110ac231a84ac3a3778 in term 1.
I20260811 02:55:25.599377 32026 raft_consensus.cc:2454] T 9cf2eb209cfa4faeae176a09a287c435 P 924ad6fccb5f44c6a9da839c8b730081 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b276426cf6d74110ac231a84ac3a3778 in term 1.
I20260811 02:55:25.599717 31643 leader_election.cc:302] T 9cf2eb209cfa4faeae176a09a287c435 P b276426cf6d74110ac231a84ac3a3778 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 93b8a8a73e364049893bd06522ab3d7c, b276426cf6d74110ac231a84ac3a3778; no voters:
I20260811 02:55:25.599927 31779 raft_consensus.cc:2781] T 9cf2eb209cfa4faeae176a09a287c435 P b276426cf6d74110ac231a84ac3a3778 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 02:55:25.600114 31779 raft_consensus.cc:686] T 9cf2eb209cfa4faeae176a09a287c435 P b276426cf6d74110ac231a84ac3a3778 [term 1 LEADER]: Becoming Leader. State: Replica: b276426cf6d74110ac231a84ac3a3778, State: Running, Role: LEADER
I20260811 02:55:25.600378 31779 consensus_queue.cc:237] T 9cf2eb209cfa4faeae176a09a287c435 P b276426cf6d74110ac231a84ac3a3778 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "93b8a8a73e364049893bd06522ab3d7c" member_type: VOTER last_known_addr { host: "127.12.138.194" port: 42231 } } peers { permanent_uuid: "924ad6fccb5f44c6a9da839c8b730081" member_type: VOTER last_known_addr { host: "127.12.138.195" port: 35627 } } peers { permanent_uuid: "b276426cf6d74110ac231a84ac3a3778" member_type: VOTER last_known_addr { host: "127.12.138.193" port: 41093 } }
I20260811 02:55:25.603623 31552 catalog_manager.cc:5495] T 9cf2eb209cfa4faeae176a09a287c435 P b276426cf6d74110ac231a84ac3a3778 reported cstate change: term changed from 0 to 1, leader changed from <none> to b276426cf6d74110ac231a84ac3a3778 (127.12.138.193). New cstate: current_term: 1 leader_uuid: "b276426cf6d74110ac231a84ac3a3778" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "93b8a8a73e364049893bd06522ab3d7c" member_type: VOTER last_known_addr { host: "127.12.138.194" port: 42231 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "924ad6fccb5f44c6a9da839c8b730081" member_type: VOTER last_known_addr { host: "127.12.138.195" port: 35627 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b276426cf6d74110ac231a84ac3a3778" member_type: VOTER last_known_addr { host: "127.12.138.193" port: 41093 } health_report { overall_health: HEALTHY } } }
W20260811 02:55:25.629122 32085 tablet.cc:2369] T 9cf2eb209cfa4faeae176a09a287c435 P 924ad6fccb5f44c6a9da839c8b730081: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 02:55:25.630403 32109 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:55:25.626547 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43014 (local address 127.12.138.193:41093)
0811 02:55:25.626873 (+ 326us) server_negotiation.cc:206] Beginning negotiation
0811 02:55:25.626880 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 02:55:25.626897 (+ 17us) server_negotiation.cc:407] Connection header received
0811 02:55:25.626999 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:55:25.627002 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:55:25.627104 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:55:25.627216 (+ 112us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 02:55:25.627455 (+ 239us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:55:25.627978 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:55:25.628756 (+ 778us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:55:25.628993 (+ 237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:55:25.629079 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:55:25.629443 (+ 364us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 02:55:25.629897 (+ 454us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 02:55:25.629993 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0811 02:55:25.630110 (+ 117us) server_negotiation.cc:299] Negotiation successful
0811 02:55:25.630161 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":226,"thread_start_us":125,"threads_started":1}
I20260811 02:55:25.634523 31876 raft_consensus.cc:1261] T 9cf2eb209cfa4faeae176a09a287c435 P 93b8a8a73e364049893bd06522ab3d7c [term 1 FOLLOWER]: Refusing update from remote peer b276426cf6d74110ac231a84ac3a3778: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 02:55:25.634523 32026 raft_consensus.cc:1261] T 9cf2eb209cfa4faeae176a09a287c435 P 924ad6fccb5f44c6a9da839c8b730081 [term 1 FOLLOWER]: Refusing update from remote peer b276426cf6d74110ac231a84ac3a3778: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 02:55:25.635254 31779 consensus_queue.cc:1035] T 9cf2eb209cfa4faeae176a09a287c435 P b276426cf6d74110ac231a84ac3a3778 [LEADER]: Connected to new peer: Peer: permanent_uuid: "924ad6fccb5f44c6a9da839c8b730081" member_type: VOTER last_known_addr { host: "127.12.138.195" port: 35627 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 02:55:25.635479 32106 consensus_queue.cc:1035] T 9cf2eb209cfa4faeae176a09a287c435 P b276426cf6d74110ac231a84ac3a3778 [LEADER]: Connected to new peer: Peer: permanent_uuid: "93b8a8a73e364049893bd06522ab3d7c" member_type: VOTER last_known_addr { host: "127.12.138.194" port: 42231 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 02:55:25.644086 32113 mvcc.cc:205] Tried to move back new op lower bound from 7317163727391838208 to 7317163727261831168. Current Snapshot: MvccSnapshot[applied={T|T < 7317163727391838208}]
I20260811 02:55:25.645568 32112 mvcc.cc:205] Tried to move back new op lower bound from 7317163727391838208 to 7317163727261831168. Current Snapshot: MvccSnapshot[applied={T|T < 7317163727391838208 or (T in {7317163727391838208})}]
I20260811 02:55:25.651744 31552 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37494:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 02:55:25.651901 31552 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37494:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 02:55:25.654407 31552 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 4ce22d68cbc247349baf03912c7e6cb6: Sending DeleteTablet for 3 replicas of tablet 9cf2eb209cfa4faeae176a09a287c435
I20260811 02:55:25.655243 31695 tablet_service.cc:1514] Processing DeleteTablet for tablet 9cf2eb209cfa4faeae176a09a287c435 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 02:55:25 UTC) from {username='kudu', principal='kudu/127.12.138.254@KRBTEST.COM'} at 127.0.0.1:43004
I20260811 02:55:25.655539 31854 tablet_service.cc:1514] Processing DeleteTablet for tablet 9cf2eb209cfa4faeae176a09a287c435 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 02:55:25 UTC) from {username='kudu', principal='kudu/127.12.138.254@KRBTEST.COM'} at 127.0.0.1:43270
I20260811 02:55:25.656220 32120 tablet_replica.cc:330] T 9cf2eb209cfa4faeae176a09a287c435 P b276426cf6d74110ac231a84ac3a3778: stopping tablet replica
I20260811 02:55:25.656296 32003 tablet_service.cc:1514] Processing DeleteTablet for tablet 9cf2eb209cfa4faeae176a09a287c435 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 02:55:25 UTC) from {username='kudu', principal='kudu/127.12.138.254@KRBTEST.COM'} at 127.0.0.1:34374
I20260811 02:55:25.656371 32120 raft_consensus.cc:2229] T 9cf2eb209cfa4faeae176a09a287c435 P b276426cf6d74110ac231a84ac3a3778 [term 1 LEADER]: Raft consensus shutting down.
I20260811 02:55:25.656706 32120 raft_consensus.cc:2258] T 9cf2eb209cfa4faeae176a09a287c435 P b276426cf6d74110ac231a84ac3a3778 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 02:55:25.656855 12843 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDawEwS/build/debug/bin/kudu with pid 31599
I20260811 02:55:25.656878 32123 tablet_replica.cc:330] T 9cf2eb209cfa4faeae176a09a287c435 P 924ad6fccb5f44c6a9da839c8b730081: stopping tablet replica
I20260811 02:55:25.656999 32121 tablet_replica.cc:330] T 9cf2eb209cfa4faeae176a09a287c435 P 93b8a8a73e364049893bd06522ab3d7c: stopping tablet replica
I20260811 02:55:25.657730 32121 raft_consensus.cc:2229] T 9cf2eb209cfa4faeae176a09a287c435 P 93b8a8a73e364049893bd06522ab3d7c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 02:55:25.658169 32121 raft_consensus.cc:2258] T 9cf2eb209cfa4faeae176a09a287c435 P 93b8a8a73e364049893bd06522ab3d7c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 02:55:25.660460 32121 ts_tablet_manager.cc:1905] T 9cf2eb209cfa4faeae176a09a287c435 P 93b8a8a73e364049893bd06522ab3d7c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 02:55:25.662019 32123 raft_consensus.cc:2229] T 9cf2eb209cfa4faeae176a09a287c435 P 924ad6fccb5f44c6a9da839c8b730081 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 02:55:25.662418 32123 raft_consensus.cc:2258] T 9cf2eb209cfa4faeae176a09a287c435 P 924ad6fccb5f44c6a9da839c8b730081 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 02:55:25.663616 32121 ts_tablet_manager.cc:1918] T 9cf2eb209cfa4faeae176a09a287c435 P 93b8a8a73e364049893bd06522ab3d7c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 02:55:25.663715 32121 log.cc:1196] T 9cf2eb209cfa4faeae176a09a287c435 P 93b8a8a73e364049893bd06522ab3d7c: Deleting WAL directory at /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786416707705494-12843-0/minicluster-data/ts-1/wal/wals/9cf2eb209cfa4faeae176a09a287c435
I20260811 02:55:25.664072 32121 ts_tablet_manager.cc:1939] T 9cf2eb209cfa4faeae176a09a287c435 P 93b8a8a73e364049893bd06522ab3d7c: Deleting consensus metadata
I20260811 02:55:25.665071 32123 ts_tablet_manager.cc:1905] T 9cf2eb209cfa4faeae176a09a287c435 P 924ad6fccb5f44c6a9da839c8b730081: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 02:55:25.665438 31537 catalog_manager.cc:4841] TS 93b8a8a73e364049893bd06522ab3d7c (127.12.138.194:42231): tablet 9cf2eb209cfa4faeae176a09a287c435 (table test-table [id=e3de53e34c3b43c7aaae8893d9fb65af]) successfully deleted
I20260811 02:55:25.667362 32123 ts_tablet_manager.cc:1918] T 9cf2eb209cfa4faeae176a09a287c435 P 924ad6fccb5f44c6a9da839c8b730081: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 02:55:25.667479 32123 log.cc:1196] T 9cf2eb209cfa4faeae176a09a287c435 P 924ad6fccb5f44c6a9da839c8b730081: Deleting WAL directory at /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786416707705494-12843-0/minicluster-data/ts-2/wal/wals/9cf2eb209cfa4faeae176a09a287c435
I20260811 02:55:25.667863 32123 ts_tablet_manager.cc:1939] T 9cf2eb209cfa4faeae176a09a287c435 P 924ad6fccb5f44c6a9da839c8b730081: Deleting consensus metadata
I20260811 02:55:25.669080 31536 catalog_manager.cc:4841] TS 924ad6fccb5f44c6a9da839c8b730081 (127.12.138.195:35627): tablet 9cf2eb209cfa4faeae176a09a287c435 (table test-table [id=e3de53e34c3b43c7aaae8893d9fb65af]) successfully deleted
W20260811 02:55:25.671238 31538 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.12.138.193:41093 (error 108)
I20260811 02:55:25.672158 12843 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDawEwS/build/debug/bin/kudu with pid 31777
I20260811 02:55:25.672335 32095 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:55:25.672034 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.138.193:41093 (local address 127.0.0.1:43016)
0811 02:55:25.672164 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0811 02:55:25.672240 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.138.193:41093: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
W20260811 02:55:25.673434 31538 catalog_manager.cc:4584] TS b276426cf6d74110ac231a84ac3a3778 (127.12.138.193:41093): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 9cf2eb209cfa4faeae176a09a287c435: Network error: Client connection negotiation failed: client connection to 127.12.138.193:41093: connect: Connection refused (error 111)
I20260811 02:55:25.682284 12843 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDawEwS/build/debug/bin/kudu with pid 31937
I20260811 02:55:25.690141 12843 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDawEwS/build/debug/bin/kudu with pid 31517
I20260811 02:55:45.699560 12843 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDawEwS/build/debug/bin/kudu
/tmp/dist-test-taskDawEwS/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786416707705494-12843-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786416707705494-12843-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786416707705494-12843-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786416707705494-12843-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.138.254:39391
--webserver_interface=127.12.138.254
--webserver_port=44325
--keytab_file=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786416707705494-12843-0/krb5kdc/kudu_127.12.138.254.keytab
--principal=kudu/127.12.138.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.138.212:42413
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.12.138.254:39391
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786416707705494-12843-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786416707705494-12843-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786416707705494-12843-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 02:55:45.824470 32125 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 02:55:45.824756 32125 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 02:55:45.824831 32125 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 02:55:45.829056 32125 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260811 02:55:45.829123 32125 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260811 02:55:45.829195 32125 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260811 02:55:45.829218 32125 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 02:55:45.829265 32125 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 02:55:45.829306 32125 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 02:55:45.829331 32125 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 02:55:45.829356 32125 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 02:55:45.834942 32125 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.138.212:42413
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786416707705494-12843-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786416707705494-12843-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.12.138.254:39391
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786416707705494-12843-0/krb5kdc/kudu_127.12.138.254.keytab
--principal=kudu/127.12.138.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.138.254:39391
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786416707705494-12843-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.12.138.254
--webserver_port=44325
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32125
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786416707705494-12843-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 02:09:49 UTC on ccycloud.quasar-fsnqqc.root.comops.site
build id 4579570
I20260811 02:55:45.836088 32125 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 02:55:45.837268 32125 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 02:55:45.846407 32133 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:55:45.846401 32131 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:55:45.846401 32130 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:55:45.847096 32125 server_base.cc:1034] running on GCE node
I20260811 02:55:45.847651 32125 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 02:55:45.848776 32125 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 02:55:45.849946 32125 hybrid_clock.cc:648] HybridClock initialized: now 1786416945849896 us; error 67 us; skew 500 ppm
Aug 11 02:55:45 dist-test-slave-9gcw krb5kdc[30885](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416945, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.138.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 02:55:45.857509 32125 init.cc:377] Logged in from keytab as kudu/127.12.138.254@KRBTEST.COM (short username kudu)
I20260811 02:55:45.858588 32125 webserver.cc:460] Webserver started at http://127.12.138.254:44325/ using document root <none> and password file <none>
I20260811 02:55:45.859153 32125 fs_manager.cc:359] Metadata directory not provided
I20260811 02:55:45.859226 32125 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 02:55:45.863432 32125 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.000s sys 0.004s
I20260811 02:55:45.865926 32140 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 02:55:45.867367 32125 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260811 02:55:45.867512 32125 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786416707705494-12843-0/minicluster-data/master-0/data,/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786416707705494-12843-0/minicluster-data/master-0/wal
uuid: "4ce22d68cbc247349baf03912c7e6cb6"
format_stamp: "Formatted at 2026-08-11 02:55:03 on dist-test-slave-9gcw"
server_key: "82b9b2173efd9043c9a0a661c2784363"
server_key_iv: "59f89185dfc326077347b28838f76d15"
server_key_version: "encryptionkey@0"
I20260811 02:55:45.868103 32125 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786416707705494-12843-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786416707705494-12843-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786416707705494-12843-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 02:55:45.883742 32125 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 02:55:45.887367 32125 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 02:55:45.887606 32125 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 02:55:45.895623 32125 rpc_server.cc:307] RPC server started. Bound to: 127.12.138.254:39391
I20260811 02:55:45.895962 32198 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.138.254:39391 every 8 connection(s)
I20260811 02:55:45.896826 32125 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786416707705494-12843-0/minicluster-data/master-0/data/info.pb
I20260811 02:55:45.897784 12843 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDawEwS/build/debug/bin/kudu as pid 32125
I20260811 02:55:45.898252 12843 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDawEwS/build/debug/bin/kudu
/tmp/dist-test-taskDawEwS/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786416707705494-12843-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786416707705494-12843-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786416707705494-12843-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786416707705494-12843-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.138.193:41093
--local_ip_for_outbound_sockets=127.12.138.193
--tserver_master_addrs=127.12.138.254:39391
--webserver_port=41777
--webserver_interface=127.12.138.193
--keytab_file=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786416707705494-12843-0/krb5kdc/kudu_127.12.138.193.keytab
--principal=kudu/127.12.138.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.138.212:42413
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786416707705494-12843-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786416707705494-12843-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786416707705494-12843-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 02:55:45.901707 32199 sys_catalog.cc:263] Verifying existing consensus state
I20260811 02:55:45.903537 32199 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4ce22d68cbc247349baf03912c7e6cb6: Bootstrap starting.
I20260811 02:55:45.922207 32199 log.cc:824] T 00000000000000000000000000000000 P 4ce22d68cbc247349baf03912c7e6cb6: Log is configured to *not* fsync() on all Append() calls
I20260811 02:55:45.935184 32199 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4ce22d68cbc247349baf03912c7e6cb6: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=8 ignored=0} mutations{seen=8 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 02:55:45.935700 32199 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4ce22d68cbc247349baf03912c7e6cb6: Bootstrap complete.
I20260811 02:55:45.940279 32199 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4ce22d68cbc247349baf03912c7e6cb6 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4ce22d68cbc247349baf03912c7e6cb6" member_type: VOTER last_known_addr { host: "127.12.138.254" port: 39391 } }
I20260811 02:55:45.940721 32199 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4ce22d68cbc247349baf03912c7e6cb6 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4ce22d68cbc247349baf03912c7e6cb6, State: Initialized, Role: FOLLOWER
I20260811 02:55:45.941315 32199 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4ce22d68cbc247349baf03912c7e6cb6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4ce22d68cbc247349baf03912c7e6cb6" member_type: VOTER last_known_addr { host: "127.12.138.254" port: 39391 } }
I20260811 02:55:45.941481 32199 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4ce22d68cbc247349baf03912c7e6cb6 [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 02:55:45.941577 32199 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4ce22d68cbc247349baf03912c7e6cb6 [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 02:55:45.941716 32199 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4ce22d68cbc247349baf03912c7e6cb6 [term 2 FOLLOWER]: Advancing to term 3
I20260811 02:55:45.943739 32199 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4ce22d68cbc247349baf03912c7e6cb6 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4ce22d68cbc247349baf03912c7e6cb6" member_type: VOTER last_known_addr { host: "127.12.138.254" port: 39391 } }
I20260811 02:55:45.944110 32199 leader_election.cc:302] T 00000000000000000000000000000000 P 4ce22d68cbc247349baf03912c7e6cb6 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4ce22d68cbc247349baf03912c7e6cb6; no voters:
I20260811 02:55:45.944499 32199 leader_election.cc:288] T 00000000000000000000000000000000 P 4ce22d68cbc247349baf03912c7e6cb6 [CANDIDATE]: Term 3 election: Requested vote from peers
I20260811 02:55:45.944597 32203 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4ce22d68cbc247349baf03912c7e6cb6 [term 3 FOLLOWER]: Leader election won for term 3
I20260811 02:55:45.944859 32203 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4ce22d68cbc247349baf03912c7e6cb6 [term 3 LEADER]: Becoming Leader. State: Replica: 4ce22d68cbc247349baf03912c7e6cb6, State: Running, Role: LEADER
I20260811 02:55:45.945210 32203 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4ce22d68cbc247349baf03912c7e6cb6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4ce22d68cbc247349baf03912c7e6cb6" member_type: VOTER last_known_addr { host: "127.12.138.254" port: 39391 } }
I20260811 02:55:45.945894 32199 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4ce22d68cbc247349baf03912c7e6cb6 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 02:55:45.946621 32204 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4ce22d68cbc247349baf03912c7e6cb6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "4ce22d68cbc247349baf03912c7e6cb6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4ce22d68cbc247349baf03912c7e6cb6" member_type: VOTER last_known_addr { host: "127.12.138.254" port: 39391 } } }
I20260811 02:55:45.946753 32204 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4ce22d68cbc247349baf03912c7e6cb6 [sys.catalog]: This master's current role is: LEADER
I20260811 02:55:45.947474 32205 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4ce22d68cbc247349baf03912c7e6cb6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4ce22d68cbc247349baf03912c7e6cb6. Latest consensus state: current_term: 3 leader_uuid: "4ce22d68cbc247349baf03912c7e6cb6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4ce22d68cbc247349baf03912c7e6cb6" member_type: VOTER last_known_addr { host: "127.12.138.254" port: 39391 } } }
I20260811 02:55:45.947623 32205 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4ce22d68cbc247349baf03912c7e6cb6 [sys.catalog]: This master's current role is: LEADER
I20260811 02:55:45.947944 32210 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 02:55:45.953042 32210 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 02:55:45.953713 32210 catalog_manager.cc:1259] Loaded cluster ID: ef117953dc34499cbebe471f50cf1592
I20260811 02:55:45.953768 32210 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 02:55:45.956261 32210 catalog_manager.cc:1504] Loading token signing keys...
I20260811 02:55:45.956991 32210 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 4ce22d68cbc247349baf03912c7e6cb6: Loaded TSK: 0
I20260811 02:55:45.957075 32210 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 4ce22d68cbc247349baf03912c7e6cb6: Loaded TSK: 1
F20260811 02:55:45.957880 32210 token_signing_key.cc:66] Check failed: _s.ok() Bad status: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7fe4b775f9ad google::LogMessage::Fail() at ??:0
@ 0x7fe4b776288a google::LogMessage::SendToLog() at ??:0
@ 0x7fe4b775f4de google::LogMessage::Flush() at ??:0
@ 0x7fe4b7760a19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7fe4ba42318d kudu::security::TokenSigningPrivateKey::TokenSigningPrivateKey() at ??:0
@ 0x7fe4ba41bf0d kudu::security::TokenSigner::ImportKeys() at ??:0
@ 0x7fe4c20c47bb kudu::master::CatalogManager::LoadTskEntries() at ??:0
@ 0x7fe4c209e99c kudu::master::CatalogManager::InitTokenSigner() at ??:0
@ 0x7fe4c209f178 _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlvE3_clEv at ??:0
@ 0x7fe4c20e0df8 _ZNSt17_Function_handlerIFN4kudu6StatusEvEZNS0_6master14CatalogManager24PrepareForLeadershipTaskEvEUlvE3_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x7fe4c20ff82b std::function<>::operator()() at ??:0
@ 0x7fe4c209eb4f _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlSt8functionIFNS_6StatusEvEERKNS_9consensus13RaftConsensusElPKcE_clES5_S9_lSB_ at ??:0
@ 0x7fe4c209ff3e kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7fe4c209c579 _ZZN4kudu6master14CatalogManager17ElectedAsLeaderCbEvENKUlvE_clEv at ??:0
@ 0x7fe4c20e0640 _ZNSt17_Function_handlerIFvvEZN4kudu6master14CatalogManager17ElectedAsLeaderCbEvEUlvE_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x61d838 std::function<>::operator()() at ??:0
@ 0x7fe4b99b13d7 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7fe4b99b1c83 _ZZN4kudu10ThreadPool12CreateThreadEvENKUlvE_clEv at ??:0
@ 0x7fe4b99b33f2 _ZNSt17_Function_handlerIFvvEZN4kudu10ThreadPool12CreateThreadEvEUlvE_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x61d838 std::function<>::operator()() at ??:0
@ 0x7fe4b99a2512 kudu::Thread::SuperviseThread() at ??:0
@ 0x7fe4bafe66db start_thread at ??:0
@ 0x7fe4b625d61f clone at ??:0
W20260811 02:55:46.049677 32201 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 02:55:46.049935 32201 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 02:55:46.049984 32201 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 02:55:46.055076 32201 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 02:55:46.055137 32201 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 02:55:46.055214 32201 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.138.193
I20260811 02:55:46.062119 32201 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.138.212:42413
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786416707705494-12843-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786416707705494-12843-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786416707705494-12843-0/krb5kdc/kudu_127.12.138.193.keytab
--principal=kudu/127.12.138.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.138.193:41093
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786416707705494-12843-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.12.138.193
--webserver_port=41777
--tserver_master_addrs=127.12.138.254:39391
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32201
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.138.193
--log_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786416707705494-12843-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 02:09:49 UTC on ccycloud.quasar-fsnqqc.root.comops.site
build id 4579570
I20260811 02:55:46.063271 32201 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 02:55:46.064335 32201 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 02:55:46.076153 32254 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:55:46.076395 32201 server_base.cc:1034] running on GCE node
W20260811 02:55:46.076143 32252 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:55:46.076203 32250 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:55:46.077060 32201 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 02:55:46.077803 32201 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 02:55:46.078987 32201 hybrid_clock.cc:648] HybridClock initialized: now 1786416946078966 us; error 41 us; skew 500 ppm
Aug 11 02:55:46 dist-test-slave-9gcw krb5kdc[30885](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416946, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.138.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 02:55:46.082042 32201 init.cc:377] Logged in from keytab as kudu/127.12.138.193@KRBTEST.COM (short username kudu)
I20260811 02:55:46.083009 32201 webserver.cc:460] Webserver started at http://127.12.138.193:41777/ using document root <none> and password file <none>
I20260811 02:55:46.083599 32201 fs_manager.cc:359] Metadata directory not provided
I20260811 02:55:46.083674 32201 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 02:55:46.088227 32201 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.000s
I20260811 02:55:46.090711 32261 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 02:55:46.092181 32201 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260811 02:55:46.092355 32201 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786416707705494-12843-0/minicluster-data/ts-0/data,/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786416707705494-12843-0/minicluster-data/ts-0/wal
uuid: "b276426cf6d74110ac231a84ac3a3778"
format_stamp: "Formatted at 2026-08-11 02:55:03 on dist-test-slave-9gcw"
server_key: "4705960a372f5dbbafe86ae8d0016d6a"
server_key_iv: "c017c9d055ebf8a2f24ec71073419281"
server_key_version: "encryptionkey@0"
I20260811 02:55:46.093014 32201 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786416707705494-12843-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786416707705494-12843-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786416707705494-12843-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 02:55:46.131942 32201 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 02:55:46.136230 32201 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 02:55:46.136488 32201 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 02:55:46.137282 32201 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 02:55:46.138727 32268 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260811 02:55:46.144997 32201 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260811 02:55:46.145124 32201 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.007s user 0.001s sys 0.000s
I20260811 02:55:46.145239 32201 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260811 02:55:46.147140 32201 ts_tablet_manager.cc:610] Registered 1 tablets
I20260811 02:55:46.147223 32201 ts_tablet_manager.cc:589] Time spent register tablets: real 0.002s user 0.000s sys 0.000s
I20260811 02:55:46.147511 32268 tablet_bootstrap.cc:492] T 9cf2eb209cfa4faeae176a09a287c435 P b276426cf6d74110ac231a84ac3a3778: Bootstrap starting.
I20260811 02:55:46.158418 32201 rpc_server.cc:307] RPC server started. Bound to: 127.12.138.193:41093
I20260811 02:55:46.158471 32396 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.138.193:41093 every 8 connection(s)
I20260811 02:55:46.159420 32201 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786416707705494-12843-0/minicluster-data/ts-0/data/info.pb
I20260811 02:55:46.167454 12843 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDawEwS/build/debug/bin/kudu as pid 32201
I20260811 02:55:46.168025 12843 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDawEwS/build/debug/bin/kudu
/tmp/dist-test-taskDawEwS/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786416707705494-12843-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786416707705494-12843-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786416707705494-12843-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786416707705494-12843-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.138.194:42231
--local_ip_for_outbound_sockets=127.12.138.194
--tserver_master_addrs=127.12.138.254:39391
--webserver_port=33595
--webserver_interface=127.12.138.194
--keytab_file=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786416707705494-12843-0/krb5kdc/kudu_127.12.138.194.keytab
--principal=kudu/127.12.138.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.138.212:42413
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786416707705494-12843-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786416707705494-12843-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786416707705494-12843-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 02:55:46.169678 32268 log.cc:824] T 9cf2eb209cfa4faeae176a09a287c435 P b276426cf6d74110ac231a84ac3a3778: Log is configured to *not* fsync() on all Append() calls
I20260811 02:55:46.179179 32268 tablet_bootstrap.cc:492] T 9cf2eb209cfa4faeae176a09a287c435 P b276426cf6d74110ac231a84ac3a3778: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 02:55:46.179651 32268 tablet_bootstrap.cc:492] T 9cf2eb209cfa4faeae176a09a287c435 P b276426cf6d74110ac231a84ac3a3778: Bootstrap complete.
I20260811 02:55:46.181088 32268 ts_tablet_manager.cc:1397] T 9cf2eb209cfa4faeae176a09a287c435 P b276426cf6d74110ac231a84ac3a3778: Time spent bootstrapping tablet: real 0.034s user 0.025s sys 0.004s
I20260811 02:55:46.184442 32268 raft_consensus.cc:348] T 9cf2eb209cfa4faeae176a09a287c435 P b276426cf6d74110ac231a84ac3a3778 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "93b8a8a73e364049893bd06522ab3d7c" member_type: VOTER last_known_addr { host: "127.12.138.194" port: 42231 } } peers { permanent_uuid: "924ad6fccb5f44c6a9da839c8b730081" member_type: VOTER last_known_addr { host: "127.12.138.195" port: 35627 } } peers { permanent_uuid: "b276426cf6d74110ac231a84ac3a3778" member_type: VOTER last_known_addr { host: "127.12.138.193" port: 41093 } }
I20260811 02:55:46.185009 32268 raft_consensus.cc:729] T 9cf2eb209cfa4faeae176a09a287c435 P b276426cf6d74110ac231a84ac3a3778 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: b276426cf6d74110ac231a84ac3a3778, State: Initialized, Role: FOLLOWER
I20260811 02:55:46.185582 32268 consensus_queue.cc:260] T 9cf2eb209cfa4faeae176a09a287c435 P b276426cf6d74110ac231a84ac3a3778 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "93b8a8a73e364049893bd06522ab3d7c" member_type: VOTER last_known_addr { host: "127.12.138.194" port: 42231 } } peers { permanent_uuid: "924ad6fccb5f44c6a9da839c8b730081" member_type: VOTER last_known_addr { host: "127.12.138.195" port: 35627 } } peers { permanent_uuid: "b276426cf6d74110ac231a84ac3a3778" member_type: VOTER last_known_addr { host: "127.12.138.193" port: 41093 } }
I20260811 02:55:46.186695 32268 ts_tablet_manager.cc:1428] T 9cf2eb209cfa4faeae176a09a287c435 P b276426cf6d74110ac231a84ac3a3778: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
W20260811 02:55:46.223228 32399 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 02:55:46.161454 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.138.254:39391 (local address 127.12.138.193:49721)
0811 02:55:46.161951 (+ 497us) negotiation.cc:107] Waiting for socket to connect
0811 02:55:46.162005 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0811 02:55:46.162834 (+ 829us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:55:46.222174 (+ 59340us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.138.254:39391: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":283,"thread_start_us":147,"threads_started":1}
W20260811 02:55:46.223883 32397 heartbeater.cc:644] Failed to heartbeat to 127.12.138.254:39391 (0 consecutive failures): Network error: Failed to ping master at 127.12.138.254:39391: Client connection negotiation failed: client connection to 127.12.138.254:39391: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260811 02:55:46.224668 32399 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:55:46.224370 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.138.254:39391 (local address 127.12.138.193:37823)
0811 02:55:46.224518 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0811 02:55:46.224598 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.138.254:39391: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
I20260811 02:55:46.225371 32399 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:55:46.225132 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.138.254:39391 (local address 127.12.138.193:46829)
0811 02:55:46.225267 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0811 02:55:46.225322 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.138.254:39391: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":70}
W20260811 02:55:46.225579 32397 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260811 02:55:46.292783 32400 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 02:55:46.293077 32400 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 02:55:46.293123 32400 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 02:55:46.297484 32400 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 02:55:46.297564 32400 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 02:55:46.297663 32400 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.138.194
I20260811 02:55:46.303321 32400 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.138.212:42413
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786416707705494-12843-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786416707705494-12843-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786416707705494-12843-0/krb5kdc/kudu_127.12.138.194.keytab
--principal=kudu/127.12.138.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.138.194:42231
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786416707705494-12843-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.12.138.194
--webserver_port=33595
--tserver_master_addrs=127.12.138.254:39391
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32400
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.138.194
--log_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786416707705494-12843-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 02:09:49 UTC on ccycloud.quasar-fsnqqc.root.comops.site
build id 4579570
I20260811 02:55:46.304423 32400 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 02:55:46.305488 32400 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 02:55:46.315564 32409 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:55:46.315574 32408 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:55:46.315574 32411 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:55:46.315843 32400 server_base.cc:1034] running on GCE node
I20260811 02:55:46.316339 32400 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 02:55:46.317000 32400 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 02:55:46.318195 32400 hybrid_clock.cc:648] HybridClock initialized: now 1786416946318182 us; error 28 us; skew 500 ppm
Aug 11 02:55:46 dist-test-slave-9gcw krb5kdc[30885](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416946, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.138.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 02:55:46.321003 32400 init.cc:377] Logged in from keytab as kudu/127.12.138.194@KRBTEST.COM (short username kudu)
I20260811 02:55:46.321928 32400 webserver.cc:460] Webserver started at http://127.12.138.194:33595/ using document root <none> and password file <none>
I20260811 02:55:46.322511 32400 fs_manager.cc:359] Metadata directory not provided
I20260811 02:55:46.322609 32400 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 02:55:46.326565 32400 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260811 02:55:46.328851 32419 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 02:55:46.330212 32400 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260811 02:55:46.330356 32400 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786416707705494-12843-0/minicluster-data/ts-1/data,/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786416707705494-12843-0/minicluster-data/ts-1/wal
uuid: "93b8a8a73e364049893bd06522ab3d7c"
format_stamp: "Formatted at 2026-08-11 02:55:04 on dist-test-slave-9gcw"
server_key: "adb77886557a128a3658d3768cfeb27d"
server_key_iv: "73633ad8d01bc30fbc2dc442c7d12812"
server_key_version: "encryptionkey@0"
I20260811 02:55:46.330958 32400 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786416707705494-12843-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786416707705494-12843-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786416707705494-12843-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 02:55:46.344448 32400 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 02:55:46.347754 32400 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 02:55:46.348001 32400 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 02:55:46.348604 32400 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 02:55:46.349520 32400 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 02:55:46.349596 32400 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 02:55:46.349671 32400 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 02:55:46.349715 32400 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 02:55:46.359659 32400 rpc_server.cc:307] RPC server started. Bound to: 127.12.138.194:42231
I20260811 02:55:46.359696 32551 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.138.194:42231 every 8 connection(s)
I20260811 02:55:46.360720 32400 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786416707705494-12843-0/minicluster-data/ts-1/data/info.pb
I20260811 02:55:46.364663 32554 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:55:46.362957 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.138.254:39391 (local address 127.12.138.194:38871)
0811 02:55:46.363857 (+ 900us) negotiation.cc:107] Waiting for socket to connect
0811 02:55:46.363964 (+ 107us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.138.254:39391: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":695,"thread_start_us":534,"threads_started":1}
W20260811 02:55:46.365314 32552 heartbeater.cc:644] Failed to heartbeat to 127.12.138.254:39391 (0 consecutive failures): Network error: Failed to ping master at 127.12.138.254:39391: Client connection negotiation failed: client connection to 127.12.138.254:39391: connect: Connection refused (error 111)
I20260811 02:55:46.365933 32554 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:55:46.365639 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.138.254:39391 (local address 127.12.138.194:44365)
0811 02:55:46.365795 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0811 02:55:46.365865 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.138.254:39391: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":66}
I20260811 02:55:46.366720 32554 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:55:46.366419 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.138.254:39391 (local address 127.12.138.194:57235)
0811 02:55:46.366551 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0811 02:55:46.366599 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.138.254:39391: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":64}
W20260811 02:55:46.366941 32552 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260811 02:55:46.369419 12843 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDawEwS/build/debug/bin/kudu as pid 32400
I20260811 02:55:46.369880 12843 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDawEwS/build/debug/bin/kudu
/tmp/dist-test-taskDawEwS/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786416707705494-12843-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786416707705494-12843-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786416707705494-12843-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786416707705494-12843-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.138.195:35627
--local_ip_for_outbound_sockets=127.12.138.195
--tserver_master_addrs=127.12.138.254:39391
--webserver_port=37877
--webserver_interface=127.12.138.195
--keytab_file=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786416707705494-12843-0/krb5kdc/kudu_127.12.138.195.keytab
--principal=kudu/127.12.138.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.138.212:42413
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786416707705494-12843-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786416707705494-12843-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786416707705494-12843-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 02:55:46.486584 32555 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 02:55:46.486855 32555 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 02:55:46.486900 32555 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 02:55:46.491592 32555 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 02:55:46.491667 32555 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 02:55:46.491729 32555 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.138.195
I20260811 02:55:46.497393 32555 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.138.212:42413
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786416707705494-12843-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786416707705494-12843-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786416707705494-12843-0/krb5kdc/kudu_127.12.138.195.keytab
--principal=kudu/127.12.138.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.138.195:35627
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786416707705494-12843-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.12.138.195
--webserver_port=37877
--tserver_master_addrs=127.12.138.254:39391
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32555
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.138.195
--log_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786416707705494-12843-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 02:09:49 UTC on ccycloud.quasar-fsnqqc.root.comops.site
build id 4579570
I20260811 02:55:46.498530 32555 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 02:55:46.499574 32555 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 02:55:46.509662 32560 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:55:46.509670 32563 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:55:46.509662 32561 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:55:46.510056 32555 server_base.cc:1034] running on GCE node
I20260811 02:55:46.510463 32555 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 02:55:46.511123 32555 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 02:55:46.512346 32555 hybrid_clock.cc:648] HybridClock initialized: now 1786416946512320 us; error 44 us; skew 500 ppm
Aug 11 02:55:46 dist-test-slave-9gcw krb5kdc[30885](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416946, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.138.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 02:55:46.515295 32555 init.cc:377] Logged in from keytab as kudu/127.12.138.195@KRBTEST.COM (short username kudu)
I20260811 02:55:46.516139 32555 webserver.cc:460] Webserver started at http://127.12.138.195:37877/ using document root <none> and password file <none>
I20260811 02:55:46.516674 32555 fs_manager.cc:359] Metadata directory not provided
I20260811 02:55:46.516737 32555 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 02:55:46.520640 32555 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.002s sys 0.000s
I20260811 02:55:46.522949 32572 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 02:55:46.524169 32555 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260811 02:55:46.524296 32555 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786416707705494-12843-0/minicluster-data/ts-2/data,/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786416707705494-12843-0/minicluster-data/ts-2/wal
uuid: "924ad6fccb5f44c6a9da839c8b730081"
format_stamp: "Formatted at 2026-08-11 02:55:04 on dist-test-slave-9gcw"
server_key: "a63d3bf5d003f94c5a55c3705422df2e"
server_key_iv: "436cefcc8a36da9bf0941d370ca47a77"
server_key_version: "encryptionkey@0"
I20260811 02:55:46.524855 32555 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786416707705494-12843-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786416707705494-12843-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786416707705494-12843-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 02:55:46.546121 32555 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 02:55:46.549707 32555 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 02:55:46.549925 32555 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 02:55:46.550556 32555 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 02:55:46.551478 32555 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 02:55:46.551529 32555 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 02:55:46.551600 32555 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 02:55:46.551640 32555 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 02:55:46.560931 32555 rpc_server.cc:307] RPC server started. Bound to: 127.12.138.195:35627
I20260811 02:55:46.560969 32697 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.138.195:35627 every 8 connection(s)
I20260811 02:55:46.561985 32555 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786416707705494-12843-0/minicluster-data/ts-2/data/info.pb
I20260811 02:55:46.565575 32700 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:55:46.564070 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.138.254:39391 (local address 127.12.138.195:38475)
0811 02:55:46.564740 (+ 670us) negotiation.cc:107] Waiting for socket to connect
0811 02:55:46.564847 (+ 107us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.138.254:39391: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":477,"thread_start_us":118,"threads_started":1}
W20260811 02:55:46.566045 32698 heartbeater.cc:644] Failed to heartbeat to 127.12.138.254:39391 (0 consecutive failures): Network error: Failed to ping master at 127.12.138.254:39391: Client connection negotiation failed: client connection to 127.12.138.254:39391: connect: Connection refused (error 111)
I20260811 02:55:46.566641 32700 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:55:46.566421 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.138.254:39391 (local address 127.12.138.195:57765)
0811 02:55:46.566536 (+ 115us) negotiation.cc:107] Waiting for socket to connect
0811 02:55:46.566589 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.138.254:39391: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
I20260811 02:55:46.567268 32700 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:55:46.567041 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.138.254:39391 (local address 127.12.138.195:58219)
0811 02:55:46.567141 (+ 100us) negotiation.cc:107] Waiting for socket to connect
0811 02:55:46.567186 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.138.254:39391: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
W20260811 02:55:46.567440 32698 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260811 02:55:46.568331 12843 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDawEwS/build/debug/bin/kudu as pid 32555
I20260811 02:55:47.226663 32702 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:55:47.226218 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.138.254:39391 (local address 127.12.138.193:60435)
0811 02:55:47.226480 (+ 262us) negotiation.cc:107] Waiting for socket to connect
0811 02:55:47.226541 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.138.254:39391: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":187,"thread_start_us":85,"threads_started":1}
I20260811 02:55:47.368047 32703 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:55:47.367596 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.138.254:39391 (local address 127.12.138.194:56769)
0811 02:55:47.367880 (+ 284us) negotiation.cc:107] Waiting for socket to connect
0811 02:55:47.367941 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.138.254:39391: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":189,"thread_start_us":89,"threads_started":1}
I20260811 02:55:47.568580 32704 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:55:47.568103 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.138.254:39391 (local address 127.12.138.195:52619)
0811 02:55:47.568399 (+ 296us) negotiation.cc:107] Waiting for socket to connect
0811 02:55:47.568465 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.138.254:39391: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":214,"thread_start_us":105,"threads_started":1}
I20260811 02:55:47.641708 32705 raft_consensus.cc:482] T 9cf2eb209cfa4faeae176a09a287c435 P b276426cf6d74110ac231a84ac3a3778 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 02:55:47.641920 32705 raft_consensus.cc:504] T 9cf2eb209cfa4faeae176a09a287c435 P b276426cf6d74110ac231a84ac3a3778 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "93b8a8a73e364049893bd06522ab3d7c" member_type: VOTER last_known_addr { host: "127.12.138.194" port: 42231 } } peers { permanent_uuid: "924ad6fccb5f44c6a9da839c8b730081" member_type: VOTER last_known_addr { host: "127.12.138.195" port: 35627 } } peers { permanent_uuid: "b276426cf6d74110ac231a84ac3a3778" member_type: VOTER last_known_addr { host: "127.12.138.193" port: 41093 } }
I20260811 02:55:47.643051 32705 leader_election.cc:288] T 9cf2eb209cfa4faeae176a09a287c435 P b276426cf6d74110ac231a84ac3a3778 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 93b8a8a73e364049893bd06522ab3d7c (127.12.138.194:42231), 924ad6fccb5f44c6a9da839c8b730081 (127.12.138.195:35627)
Aug 11 02:55:47 dist-test-slave-9gcw krb5kdc[30885](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416946, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.138.193@KRBTEST.COM for kudu/127.12.138.195@KRBTEST.COM
Aug 11 02:55:47 dist-test-slave-9gcw krb5kdc[30885](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416946, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.138.193@KRBTEST.COM for kudu/127.12.138.194@KRBTEST.COM
I20260811 02:55:47.658246 32706 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:55:47.643381 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.138.195:35627 (local address 127.12.138.193:33381)
0811 02:55:47.643702 (+ 321us) negotiation.cc:107] Waiting for socket to connect
0811 02:55:47.643716 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 02:55:47.643837 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:55:47.645512 (+ 1675us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:55:47.645520 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:55:47.646115 (+ 595us) client_negotiation.cc:189] Negotiated authn=SASL
0811 02:55:47.646611 (+ 496us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:55:47.646626 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:55:47.648379 (+ 1753us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:55:47.648382 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:55:47.649258 (+ 876us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:55:47.649269 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:55:47.649987 (+ 718us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:55:47.649990 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:55:47.650210 (+ 220us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:55:47.650982 (+ 772us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 02:55:47.650998 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0811 02:55:47.652856 (+ 1858us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 02:55:47.655555 (+ 2699us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:55:47.655561 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:55:47.655563 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:55:47.655909 (+ 346us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:55:47.656271 (+ 362us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:55:47.656275 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:55:47.656277 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:55:47.656435 (+ 158us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:55:47.657104 (+ 669us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 02:55:47.657111 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 02:55:47.657445 (+ 334us) client_negotiation.cc:769] Sending connection context
0811 02:55:47.657649 (+ 204us) client_negotiation.cc:240] Negotiation successful
0811 02:55:47.658056 (+ 407us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":260,"mutex_wait_us":226,"spinlock_wait_cycles":44032,"thread_start_us":77,"threads_started":1}
I20260811 02:55:47.658259 32708 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:55:47.643514 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.138.193:33381 (local address 127.12.138.195:35627)
0811 02:55:47.643979 (+ 465us) server_negotiation.cc:206] Beginning negotiation
0811 02:55:47.644002 (+ 23us) server_negotiation.cc:399] Waiting for connection header
0811 02:55:47.644038 (+ 36us) server_negotiation.cc:407] Connection header received
0811 02:55:47.644719 (+ 681us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:55:47.644725 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:55:47.645072 (+ 347us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:55:47.645388 (+ 316us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:55:47.647104 (+ 1716us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:55:47.648241 (+ 1137us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:55:47.649384 (+ 1143us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:55:47.649857 (+ 473us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:55:47.650037 (+ 180us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:55:47.653126 (+ 3089us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:55:47.653132 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:55:47.653429 (+ 297us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:55:47.653461 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:55:47.655430 (+ 1969us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:55:47.656028 (+ 598us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:55:47.656036 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:55:47.656046 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:55:47.656162 (+ 116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:55:47.656535 (+ 373us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:55:47.656538 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:55:47.656540 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:55:47.656982 (+ 442us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:55:47.657130 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0811 02:55:47.657764 (+ 634us) server_negotiation.cc:299] Negotiation successful
0811 02:55:47.658044 (+ 280us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":366,"thread_start_us":150,"threads_started":1}
I20260811 02:55:47.658260 32707 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:55:47.643537 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.138.193:41367 (local address 127.12.138.194:42231)
0811 02:55:47.643986 (+ 449us) server_negotiation.cc:206] Beginning negotiation
0811 02:55:47.644006 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0811 02:55:47.644038 (+ 32us) server_negotiation.cc:407] Connection header received
0811 02:55:47.644998 (+ 960us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:55:47.645005 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:55:47.645381 (+ 376us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:55:47.645683 (+ 302us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:55:47.646800 (+ 1117us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:55:47.648068 (+ 1268us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:55:47.649363 (+ 1295us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:55:47.649857 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:55:47.650045 (+ 188us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:55:47.653265 (+ 3220us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:55:47.653269 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:55:47.653282 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:55:47.653313 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:55:47.655358 (+ 2045us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:55:47.656020 (+ 662us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:55:47.656035 (+ 15us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:55:47.656039 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:55:47.656162 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:55:47.656541 (+ 379us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:55:47.656545 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:55:47.656547 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:55:47.656981 (+ 434us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:55:47.657131 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0811 02:55:47.657771 (+ 640us) server_negotiation.cc:299] Negotiation successful
0811 02:55:47.658070 (+ 299us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":335,"thread_start_us":79,"threads_started":1}
I20260811 02:55:47.658875 32702 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:55:47.643342 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.138.194:42231 (local address 127.12.138.193:41367)
0811 02:55:47.643538 (+ 196us) negotiation.cc:107] Waiting for socket to connect
0811 02:55:47.643554 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 02:55:47.643752 (+ 198us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:55:47.645652 (+ 1900us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:55:47.645655 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:55:47.646105 (+ 450us) client_negotiation.cc:189] Negotiated authn=SASL
0811 02:55:47.646610 (+ 505us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:55:47.646625 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:55:47.648244 (+ 1619us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:55:47.648247 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:55:47.649237 (+ 990us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:55:47.649247 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:55:47.649998 (+ 751us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:55:47.650003 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:55:47.650254 (+ 251us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:55:47.650935 (+ 681us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 02:55:47.650952 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 02:55:47.653046 (+ 2094us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 02:55:47.655498 (+ 2452us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:55:47.655505 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:55:47.655520 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:55:47.655897 (+ 377us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:55:47.656280 (+ 383us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:55:47.656283 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:55:47.656285 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:55:47.656441 (+ 156us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:55:47.657105 (+ 664us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 02:55:47.657111 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 02:55:47.657445 (+ 334us) client_negotiation.cc:769] Sending connection context
0811 02:55:47.657649 (+ 204us) client_negotiation.cc:240] Negotiation successful
0811 02:55:47.658758 (+ 1109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":116,"mutex_wait_us":227,"spinlock_wait_cycles":9088}
W20260811 02:55:47.661369 32263 leader_election.cc:341] T 9cf2eb209cfa4faeae176a09a287c435 P b276426cf6d74110ac231a84ac3a3778 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 93b8a8a73e364049893bd06522ab3d7c (127.12.138.194:42231): Not found: Tablet not found: 9cf2eb209cfa4faeae176a09a287c435
W20260811 02:55:47.661538 32262 leader_election.cc:341] T 9cf2eb209cfa4faeae176a09a287c435 P b276426cf6d74110ac231a84ac3a3778 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 924ad6fccb5f44c6a9da839c8b730081 (127.12.138.195:35627): Not found: Tablet not found: 9cf2eb209cfa4faeae176a09a287c435
I20260811 02:55:47.661649 32262 leader_election.cc:302] T 9cf2eb209cfa4faeae176a09a287c435 P b276426cf6d74110ac231a84ac3a3778 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b276426cf6d74110ac231a84ac3a3778; no voters: 924ad6fccb5f44c6a9da839c8b730081, 93b8a8a73e364049893bd06522ab3d7c
I20260811 02:55:47.661971 32705 raft_consensus.cc:2726] T 9cf2eb209cfa4faeae176a09a287c435 P b276426cf6d74110ac231a84ac3a3778 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260811 02:55:48.227921 32709 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:55:48.227484 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.138.254:39391 (local address 127.12.138.193:34939)
0811 02:55:48.227758 (+ 274us) negotiation.cc:107] Waiting for socket to connect
0811 02:55:48.227820 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.138.254:39391: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":200,"thread_start_us":96,"threads_started":1}
I20260811 02:55:48.369334 32710 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:55:48.368843 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.138.254:39391 (local address 127.12.138.194:58881)
0811 02:55:48.369138 (+ 295us) negotiation.cc:107] Waiting for socket to connect
0811 02:55:48.369217 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.138.254:39391: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":219,"thread_start_us":99,"threads_started":1}
I20260811 02:55:48.569995 32712 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:55:48.569440 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.138.254:39391 (local address 127.12.138.195:40977)
0811 02:55:48.569827 (+ 387us) negotiation.cc:107] Waiting for socket to connect
0811 02:55:48.569896 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.138.254:39391: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":296,"thread_start_us":84,"threads_started":1}
I20260811 02:55:49.229408 32713 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:55:49.228725 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.138.254:39391 (local address 127.12.138.193:37961)
0811 02:55:49.229235 (+ 510us) negotiation.cc:107] Waiting for socket to connect
0811 02:55:49.229299 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.138.254:39391: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":307,"thread_start_us":76,"threads_started":1}
I20260811 02:55:49.370605 32714 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:55:49.370131 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.138.254:39391 (local address 127.12.138.194:36987)
0811 02:55:49.370434 (+ 303us) negotiation.cc:107] Waiting for socket to connect
0811 02:55:49.370491 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.138.254:39391: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":222,"thread_start_us":108,"threads_started":1}
I20260811 02:55:49.571377 32715 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:55:49.570901 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.138.254:39391 (local address 127.12.138.195:54243)
0811 02:55:49.571188 (+ 287us) negotiation.cc:107] Waiting for socket to connect
0811 02:55:49.571255 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.138.254:39391: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":197,"thread_start_us":111,"threads_started":1}
I20260811 02:55:49.715451 32716 raft_consensus.cc:482] T 9cf2eb209cfa4faeae176a09a287c435 P b276426cf6d74110ac231a84ac3a3778 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 02:55:49.715586 32716 raft_consensus.cc:504] T 9cf2eb209cfa4faeae176a09a287c435 P b276426cf6d74110ac231a84ac3a3778 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "93b8a8a73e364049893bd06522ab3d7c" member_type: VOTER last_known_addr { host: "127.12.138.194" port: 42231 } } peers { permanent_uuid: "924ad6fccb5f44c6a9da839c8b730081" member_type: VOTER last_known_addr { host: "127.12.138.195" port: 35627 } } peers { permanent_uuid: "b276426cf6d74110ac231a84ac3a3778" member_type: VOTER last_known_addr { host: "127.12.138.193" port: 41093 } }
I20260811 02:55:49.716084 32716 leader_election.cc:288] T 9cf2eb209cfa4faeae176a09a287c435 P b276426cf6d74110ac231a84ac3a3778 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 93b8a8a73e364049893bd06522ab3d7c (127.12.138.194:42231), 924ad6fccb5f44c6a9da839c8b730081 (127.12.138.195:35627)
W20260811 02:55:49.717062 32263 leader_election.cc:341] T 9cf2eb209cfa4faeae176a09a287c435 P b276426cf6d74110ac231a84ac3a3778 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 93b8a8a73e364049893bd06522ab3d7c (127.12.138.194:42231): Not found: Tablet not found: 9cf2eb209cfa4faeae176a09a287c435
W20260811 02:55:49.717296 32262 leader_election.cc:341] T 9cf2eb209cfa4faeae176a09a287c435 P b276426cf6d74110ac231a84ac3a3778 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 924ad6fccb5f44c6a9da839c8b730081 (127.12.138.195:35627): Not found: Tablet not found: 9cf2eb209cfa4faeae176a09a287c435
I20260811 02:55:49.717411 32262 leader_election.cc:302] T 9cf2eb209cfa4faeae176a09a287c435 P b276426cf6d74110ac231a84ac3a3778 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b276426cf6d74110ac231a84ac3a3778; no voters: 924ad6fccb5f44c6a9da839c8b730081, 93b8a8a73e364049893bd06522ab3d7c
I20260811 02:55:49.717550 32716 raft_consensus.cc:2726] T 9cf2eb209cfa4faeae176a09a287c435 P b276426cf6d74110ac231a84ac3a3778 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260811 02:55:50.230844 32717 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:55:50.230361 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.138.254:39391 (local address 127.12.138.193:45635)
0811 02:55:50.230666 (+ 305us) negotiation.cc:107] Waiting for socket to connect
0811 02:55:50.230734 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.138.254:39391: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":213,"thread_start_us":87,"threads_started":1}
I20260811 02:55:50.372032 32718 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:55:50.371498 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.138.254:39391 (local address 127.12.138.194:53117)
0811 02:55:50.371863 (+ 365us) negotiation.cc:107] Waiting for socket to connect
0811 02:55:50.371930 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.138.254:39391: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":278,"thread_start_us":87,"threads_started":1}
I20260811 02:55:50.572791 32719 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:55:50.572302 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.138.254:39391 (local address 127.12.138.195:38225)
0811 02:55:50.572605 (+ 303us) negotiation.cc:107] Waiting for socket to connect
0811 02:55:50.572671 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.138.254:39391: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":213,"thread_start_us":95,"threads_started":1}
I20260811 02:55:51.232532 32720 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:55:51.231905 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.138.254:39391 (local address 127.12.138.193:51667)
0811 02:55:51.232331 (+ 426us) negotiation.cc:107] Waiting for socket to connect
0811 02:55:51.232401 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.138.254:39391: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":338,"thread_start_us":128,"threads_started":1}
I20260811 02:55:51.373518 32721 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:55:51.372935 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.138.254:39391 (local address 127.12.138.194:37739)
0811 02:55:51.373352 (+ 417us) negotiation.cc:107] Waiting for socket to connect
0811 02:55:51.373419 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.138.254:39391: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":339,"thread_start_us":108,"threads_started":1}
I20260811 02:55:51.574561 32722 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:55:51.573804 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.138.254:39391 (local address 127.12.138.195:46867)
0811 02:55:51.574258 (+ 454us) negotiation.cc:107] Waiting for socket to connect
0811 02:55:51.574363 (+ 105us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.138.254:39391: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":295,"thread_start_us":87,"threads_started":1}
I20260811 02:55:51.842041 32723 raft_consensus.cc:482] T 9cf2eb209cfa4faeae176a09a287c435 P b276426cf6d74110ac231a84ac3a3778 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 02:55:51.842185 32723 raft_consensus.cc:504] T 9cf2eb209cfa4faeae176a09a287c435 P b276426cf6d74110ac231a84ac3a3778 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "93b8a8a73e364049893bd06522ab3d7c" member_type: VOTER last_known_addr { host: "127.12.138.194" port: 42231 } } peers { permanent_uuid: "924ad6fccb5f44c6a9da839c8b730081" member_type: VOTER last_known_addr { host: "127.12.138.195" port: 35627 } } peers { permanent_uuid: "b276426cf6d74110ac231a84ac3a3778" member_type: VOTER last_known_addr { host: "127.12.138.193" port: 41093 } }
I20260811 02:55:51.842597 32723 leader_election.cc:288] T 9cf2eb209cfa4faeae176a09a287c435 P b276426cf6d74110ac231a84ac3a3778 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 93b8a8a73e364049893bd06522ab3d7c (127.12.138.194:42231), 924ad6fccb5f44c6a9da839c8b730081 (127.12.138.195:35627)
W20260811 02:55:51.843497 32263 leader_election.cc:341] T 9cf2eb209cfa4faeae176a09a287c435 P b276426cf6d74110ac231a84ac3a3778 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 93b8a8a73e364049893bd06522ab3d7c (127.12.138.194:42231): Not found: Tablet not found: 9cf2eb209cfa4faeae176a09a287c435
W20260811 02:55:51.843586 32262 leader_election.cc:341] T 9cf2eb209cfa4faeae176a09a287c435 P b276426cf6d74110ac231a84ac3a3778 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 924ad6fccb5f44c6a9da839c8b730081 (127.12.138.195:35627): Not found: Tablet not found: 9cf2eb209cfa4faeae176a09a287c435
I20260811 02:55:51.843640 32263 leader_election.cc:302] T 9cf2eb209cfa4faeae176a09a287c435 P b276426cf6d74110ac231a84ac3a3778 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b276426cf6d74110ac231a84ac3a3778; no voters: 924ad6fccb5f44c6a9da839c8b730081, 93b8a8a73e364049893bd06522ab3d7c
I20260811 02:55:51.843777 32723 raft_consensus.cc:2726] T 9cf2eb209cfa4faeae176a09a287c435 P b276426cf6d74110ac231a84ac3a3778 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260811 02:55:52.234069 32724 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:55:52.233575 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.138.254:39391 (local address 127.12.138.193:50541)
0811 02:55:52.233899 (+ 324us) negotiation.cc:107] Waiting for socket to connect
0811 02:55:52.233965 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.138.254:39391: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":235,"thread_start_us":100,"threads_started":1}
I20260811 02:55:52.375041 32725 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:55:52.374565 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.138.254:39391 (local address 127.12.138.194:38581)
0811 02:55:52.374871 (+ 306us) negotiation.cc:107] Waiting for socket to connect
0811 02:55:52.374938 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.138.254:39391: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":221,"thread_start_us":89,"threads_started":1}
I20260811 02:55:52.576092 32726 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:55:52.575601 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.138.254:39391 (local address 127.12.138.195:55685)
0811 02:55:52.575923 (+ 322us) negotiation.cc:107] Waiting for socket to connect
0811 02:55:52.575993 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.138.254:39391: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":236,"thread_start_us":91,"threads_started":1}
I20260811 02:55:53.235420 32728 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:55:53.234943 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.138.254:39391 (local address 127.12.138.193:45099)
0811 02:55:53.235238 (+ 295us) negotiation.cc:107] Waiting for socket to connect
0811 02:55:53.235302 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.138.254:39391: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":208,"thread_start_us":78,"threads_started":1}
I20260811 02:55:53.376405 32729 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:55:53.375922 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.138.254:39391 (local address 127.12.138.194:49837)
0811 02:55:53.376242 (+ 320us) negotiation.cc:107] Waiting for socket to connect
0811 02:55:53.376306 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.138.254:39391: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":246,"thread_start_us":95,"threads_started":1}
I20260811 02:55:53.577610 32730 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:55:53.576988 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.138.254:39391 (local address 127.12.138.195:35457)
0811 02:55:53.577454 (+ 466us) negotiation.cc:107] Waiting for socket to connect
0811 02:55:53.577513 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.138.254:39391: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":392,"thread_start_us":162,"threads_started":1}
I20260811 02:55:54.236544 32731 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:55:54.236140 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.138.254:39391 (local address 127.12.138.193:48467)
0811 02:55:54.236385 (+ 245us) negotiation.cc:107] Waiting for socket to connect
0811 02:55:54.236441 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.138.254:39391: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":175,"thread_start_us":75,"threads_started":1}
I20260811 02:55:54.377810 32732 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:55:54.377353 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.138.254:39391 (local address 127.12.138.194:35747)
0811 02:55:54.377640 (+ 287us) negotiation.cc:107] Waiting for socket to connect
0811 02:55:54.377706 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.138.254:39391: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":205,"thread_start_us":84,"threads_started":1}
I20260811 02:55:54.578943 32733 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:55:54.578395 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.138.254:39391 (local address 127.12.138.195:42011)
0811 02:55:54.578771 (+ 376us) negotiation.cc:107] Waiting for socket to connect
0811 02:55:54.578837 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.138.254:39391: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":296,"thread_start_us":94,"threads_started":1}
I20260811 02:55:55.238129 32734 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:55:55.237460 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.138.254:39391 (local address 127.12.138.193:44673)
0811 02:55:55.237932 (+ 472us) negotiation.cc:107] Waiting for socket to connect
0811 02:55:55.237995 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.138.254:39391: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":378,"thread_start_us":102,"threads_started":1}
I20260811 02:55:55.379153 32735 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:55:55.378649 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.138.254:39391 (local address 127.12.138.194:39371)
0811 02:55:55.378982 (+ 333us) negotiation.cc:107] Waiting for socket to connect
0811 02:55:55.379051 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.138.254:39391: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":245,"thread_start_us":100,"threads_started":1}
I20260811 02:55:55.381570 32736 raft_consensus.cc:482] T 9cf2eb209cfa4faeae176a09a287c435 P b276426cf6d74110ac231a84ac3a3778 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 02:55:55.381724 32736 raft_consensus.cc:504] T 9cf2eb209cfa4faeae176a09a287c435 P b276426cf6d74110ac231a84ac3a3778 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "93b8a8a73e364049893bd06522ab3d7c" member_type: VOTER last_known_addr { host: "127.12.138.194" port: 42231 } } peers { permanent_uuid: "924ad6fccb5f44c6a9da839c8b730081" member_type: VOTER last_known_addr { host: "127.12.138.195" port: 35627 } } peers { permanent_uuid: "b276426cf6d74110ac231a84ac3a3778" member_type: VOTER last_known_addr { host: "127.12.138.193" port: 41093 } }
I20260811 02:55:55.382200 32736 leader_election.cc:288] T 9cf2eb209cfa4faeae176a09a287c435 P b276426cf6d74110ac231a84ac3a3778 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 93b8a8a73e364049893bd06522ab3d7c (127.12.138.194:42231), 924ad6fccb5f44c6a9da839c8b730081 (127.12.138.195:35627)
W20260811 02:55:55.382972 32263 leader_election.cc:341] T 9cf2eb209cfa4faeae176a09a287c435 P b276426cf6d74110ac231a84ac3a3778 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 93b8a8a73e364049893bd06522ab3d7c (127.12.138.194:42231): Not found: Tablet not found: 9cf2eb209cfa4faeae176a09a287c435
W20260811 02:55:55.383113 32262 leader_election.cc:341] T 9cf2eb209cfa4faeae176a09a287c435 P b276426cf6d74110ac231a84ac3a3778 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 924ad6fccb5f44c6a9da839c8b730081 (127.12.138.195:35627): Not found: Tablet not found: 9cf2eb209cfa4faeae176a09a287c435
I20260811 02:55:55.383217 32262 leader_election.cc:302] T 9cf2eb209cfa4faeae176a09a287c435 P b276426cf6d74110ac231a84ac3a3778 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b276426cf6d74110ac231a84ac3a3778; no voters: 924ad6fccb5f44c6a9da839c8b730081, 93b8a8a73e364049893bd06522ab3d7c
I20260811 02:55:55.383379 32736 raft_consensus.cc:2726] T 9cf2eb209cfa4faeae176a09a287c435 P b276426cf6d74110ac231a84ac3a3778 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260811 02:55:55.580447 32737 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:55:55.579943 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.138.254:39391 (local address 127.12.138.195:48921)
0811 02:55:55.580287 (+ 344us) negotiation.cc:107] Waiting for socket to connect
0811 02:55:55.580350 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.138.254:39391: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":251,"thread_start_us":132,"threads_started":1}
I20260811 02:55:56.239596 32738 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:55:56.239175 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.138.254:39391 (local address 127.12.138.193:34523)
0811 02:55:56.239432 (+ 257us) negotiation.cc:107] Waiting for socket to connect
0811 02:55:56.239492 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.138.254:39391: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":182,"thread_start_us":83,"threads_started":1}
I20260811 02:55:56.380661 32739 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:55:56.380205 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.138.254:39391 (local address 127.12.138.194:34935)
0811 02:55:56.380497 (+ 292us) negotiation.cc:107] Waiting for socket to connect
0811 02:55:56.380559 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.138.254:39391: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":214,"thread_start_us":91,"threads_started":1}
I20260811 02:55:56.581887 32740 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:55:56.581440 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.138.254:39391 (local address 127.12.138.195:35715)
0811 02:55:56.581723 (+ 283us) negotiation.cc:107] Waiting for socket to connect
0811 02:55:56.581786 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.138.254:39391: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":200,"thread_start_us":77,"threads_started":1}
I20260811 02:55:57.240967 32741 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:55:57.240474 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.138.254:39391 (local address 127.12.138.193:38735)
0811 02:55:57.240792 (+ 318us) negotiation.cc:107] Waiting for socket to connect
0811 02:55:57.240856 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.138.254:39391: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":229,"thread_start_us":77,"threads_started":1}
I20260811 02:55:57.382010 32742 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:55:57.381507 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.138.254:39391 (local address 127.12.138.194:58099)
0811 02:55:57.381849 (+ 342us) negotiation.cc:107] Waiting for socket to connect
0811 02:55:57.381912 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.138.254:39391: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":220,"thread_start_us":82,"threads_started":1}
I20260811 02:55:57.583077 32743 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:55:57.582633 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.138.254:39391 (local address 127.12.138.195:34041)
0811 02:55:57.582917 (+ 284us) negotiation.cc:107] Waiting for socket to connect
0811 02:55:57.582980 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.138.254:39391: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":204,"thread_start_us":97,"threads_started":1}
I20260811 02:55:58.242307 32744 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:55:58.241846 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.138.254:39391 (local address 127.12.138.193:54303)
0811 02:55:58.242138 (+ 292us) negotiation.cc:107] Waiting for socket to connect
0811 02:55:58.242199 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.138.254:39391: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":207,"thread_start_us":82,"threads_started":1}
I20260811 02:55:58.383308 32746 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:55:58.382855 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.138.254:39391 (local address 127.12.138.194:59929)
0811 02:55:58.383145 (+ 290us) negotiation.cc:107] Waiting for socket to connect
0811 02:55:58.383209 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.138.254:39391: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":213,"thread_start_us":95,"threads_started":1}
I20260811 02:55:58.584545 32747 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:55:58.584038 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.138.254:39391 (local address 127.12.138.195:50373)
0811 02:55:58.584349 (+ 311us) negotiation.cc:107] Waiting for socket to connect
0811 02:55:58.584408 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.138.254:39391: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":222,"thread_start_us":126,"threads_started":1}
I20260811 02:55:59.243649 32748 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:55:59.243189 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.138.254:39391 (local address 127.12.138.193:38213)
0811 02:55:59.243482 (+ 293us) negotiation.cc:107] Waiting for socket to connect
0811 02:55:59.243545 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.138.254:39391: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":209,"thread_start_us":85,"threads_started":1}
I20260811 02:55:59.384575 32749 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:55:59.384144 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.138.254:39391 (local address 127.12.138.194:38983)
0811 02:55:59.384412 (+ 268us) negotiation.cc:107] Waiting for socket to connect
0811 02:55:59.384475 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.138.254:39391: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":191,"thread_start_us":86,"threads_started":1}
I20260811 02:55:59.585901 32750 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:55:59.585471 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.138.254:39391 (local address 127.12.138.195:44343)
0811 02:55:59.585741 (+ 270us) negotiation.cc:107] Waiting for socket to connect
0811 02:55:59.585804 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.138.254:39391: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":188,"thread_start_us":77,"threads_started":1}
I20260811 02:56:00.240178 32751 raft_consensus.cc:482] T 9cf2eb209cfa4faeae176a09a287c435 P b276426cf6d74110ac231a84ac3a3778 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 02:56:00.240361 32751 raft_consensus.cc:504] T 9cf2eb209cfa4faeae176a09a287c435 P b276426cf6d74110ac231a84ac3a3778 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "93b8a8a73e364049893bd06522ab3d7c" member_type: VOTER last_known_addr { host: "127.12.138.194" port: 42231 } } peers { permanent_uuid: "924ad6fccb5f44c6a9da839c8b730081" member_type: VOTER last_known_addr { host: "127.12.138.195" port: 35627 } } peers { permanent_uuid: "b276426cf6d74110ac231a84ac3a3778" member_type: VOTER last_known_addr { host: "127.12.138.193" port: 41093 } }
I20260811 02:56:00.240832 32751 leader_election.cc:288] T 9cf2eb209cfa4faeae176a09a287c435 P b276426cf6d74110ac231a84ac3a3778 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 93b8a8a73e364049893bd06522ab3d7c (127.12.138.194:42231), 924ad6fccb5f44c6a9da839c8b730081 (127.12.138.195:35627)
W20260811 02:56:00.241895 32263 leader_election.cc:341] T 9cf2eb209cfa4faeae176a09a287c435 P b276426cf6d74110ac231a84ac3a3778 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 93b8a8a73e364049893bd06522ab3d7c (127.12.138.194:42231): Not found: Tablet not found: 9cf2eb209cfa4faeae176a09a287c435
W20260811 02:56:00.242017 32262 leader_election.cc:341] T 9cf2eb209cfa4faeae176a09a287c435 P b276426cf6d74110ac231a84ac3a3778 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 924ad6fccb5f44c6a9da839c8b730081 (127.12.138.195:35627): Not found: Tablet not found: 9cf2eb209cfa4faeae176a09a287c435
I20260811 02:56:00.242116 32262 leader_election.cc:302] T 9cf2eb209cfa4faeae176a09a287c435 P b276426cf6d74110ac231a84ac3a3778 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b276426cf6d74110ac231a84ac3a3778; no voters: 924ad6fccb5f44c6a9da839c8b730081, 93b8a8a73e364049893bd06522ab3d7c
I20260811 02:56:00.242285 32751 raft_consensus.cc:2726] T 9cf2eb209cfa4faeae176a09a287c435 P b276426cf6d74110ac231a84ac3a3778 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260811 02:56:00.244817 32753 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:56:00.244393 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.138.254:39391 (local address 127.12.138.193:51503)
0811 02:56:00.244642 (+ 249us) negotiation.cc:107] Waiting for socket to connect
0811 02:56:00.244717 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.138.254:39391: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":174,"thread_start_us":98,"threads_started":1}
I20260811 02:56:00.385910 32754 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:56:00.385444 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.138.254:39391 (local address 127.12.138.194:37281)
0811 02:56:00.385712 (+ 268us) negotiation.cc:107] Waiting for socket to connect
0811 02:56:00.385779 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.138.254:39391: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":190,"thread_start_us":85,"threads_started":1}
I20260811 02:56:00.587203 32755 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:56:00.586754 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.138.254:39391 (local address 127.12.138.195:43065)
0811 02:56:00.587041 (+ 287us) negotiation.cc:107] Waiting for socket to connect
0811 02:56:00.587103 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.138.254:39391: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":213,"thread_start_us":105,"threads_started":1}
I20260811 02:56:01.246582 32756 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:56:01.246004 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.138.254:39391 (local address 127.12.138.193:60245)
0811 02:56:01.246341 (+ 337us) negotiation.cc:107] Waiting for socket to connect
0811 02:56:01.246406 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.138.254:39391: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":237,"thread_start_us":140,"threads_started":1}
I20260811 02:56:01.387598 32757 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:56:01.387118 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.138.254:39391 (local address 127.12.138.194:56353)
0811 02:56:01.387426 (+ 308us) negotiation.cc:107] Waiting for socket to connect
0811 02:56:01.387493 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.138.254:39391: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":221,"thread_start_us":91,"threads_started":1}
I20260811 02:56:01.577705 12843 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDawEwS/build/debug/bin/kudu with pid 32201
I20260811 02:56:01.588873 12843 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDawEwS/build/debug/bin/kudu with pid 32400
I20260811 02:56:01.589079 32758 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:56:01.588413 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.138.254:39391 (local address 127.12.138.195:45979)
0811 02:56:01.588841 (+ 428us) negotiation.cc:107] Waiting for socket to connect
0811 02:56:01.588898 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.138.254:39391: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":335,"thread_start_us":88,"threads_started":1}
I20260811 02:56:01.598130 12843 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDawEwS/build/debug/bin/kudu with pid 32555
2026-08-11T02:56:01Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedTSK (59680 ms)
[----------] 30 tests from SecurityITest (253890 ms total)
[----------] 6 tests from EncryptionPolicyTest
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/0
Loading random data
Initializing database '/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786416707705494-12843-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 02:56:01 dist-test-slave-9gcw krb5kdc[32761](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 02:56:01 dist-test-slave-9gcw krb5kdc[32761](info): set up 2 sockets
Aug 11 02:56:01 dist-test-slave-9gcw krb5kdc[32761](info): commencing operation
krb5kdc: starting...
W20260811 02:56:02.774855 12843 mini_kdc.cc:121] Time spent starting KDC: real 1.147s user 0.004s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786416707705494-12843-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786416707705494-12843-0/krb5kdc/test-user.keytab.
Aug 11 02:56:02 dist-test-slave-9gcw krb5kdc[32761](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416962, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T02:56:02Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T02:56:02Z Disabled control of system clock
WARNING: no policy specified for kudu/127.12.138.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.138.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.138.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786416707705494-12843-0/krb5kdc/kudu_127.12.138.254.keytab.
Entry for principal kudu/127.12.138.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786416707705494-12843-0/krb5kdc/kudu_127.12.138.254.keytab.
I20260811 02:56:02.917424 12843 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDawEwS/build/debug/bin/kudu
/tmp/dist-test-taskDawEwS/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786416707705494-12843-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786416707705494-12843-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786416707705494-12843-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786416707705494-12843-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.138.254:40905
--webserver_interface=127.12.138.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786416707705494-12843-0/krb5kdc/kudu_127.12.138.254.keytab
--principal=kudu/127.12.138.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.138.212:38789
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.12.138.254:40905
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786416707705494-12843-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786416707705494-12843-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786416707705494-12843-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 02:56:03.047926 307 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 02:56:03.048298 307 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 02:56:03.048350 307 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 02:56:03.053014 307 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 02:56:03.053107 307 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 02:56:03.053129 307 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 02:56:03.053220 307 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 02:56:03.053239 307 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 02:56:03.059335 307 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.138.212:38789
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786416707705494-12843-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786416707705494-12843-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.12.138.254:40905
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786416707705494-12843-0/krb5kdc/kudu_127.12.138.254.keytab
--principal=kudu/127.12.138.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.138.254:40905
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786416707705494-12843-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.12.138.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.307
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786416707705494-12843-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 02:09:49 UTC on ccycloud.quasar-fsnqqc.root.comops.site
build id 4579570
I20260811 02:56:03.060554 307 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 02:56:03.061772 307 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 02:56:03.071555 316 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:56:03.071595 315 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:56:03.071658 318 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:56:03.071590 307 server_base.cc:1034] running on GCE node
I20260811 02:56:03.072532 307 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 02:56:03.073769 307 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 02:56:03.074967 307 hybrid_clock.cc:648] HybridClock initialized: now 1786416963074943 us; error 46 us; skew 500 ppm
Aug 11 02:56:03 dist-test-slave-9gcw krb5kdc[32761](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416963, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.138.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 02:56:03.077966 307 init.cc:377] Logged in from keytab as kudu/127.12.138.254@KRBTEST.COM (short username kudu)
I20260811 02:56:03.078938 307 webserver.cc:460] Webserver started at http://127.12.138.254:39333/ using document root <none> and password file <none>
I20260811 02:56:03.079517 307 fs_manager.cc:359] Metadata directory not provided
I20260811 02:56:03.079645 307 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 02:56:03.079900 307 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 02:56:03.082073 307 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786416707705494-12843-0/minicluster-data/master-0/data/instance:
uuid: "4ec9fb2d07bf4bb38266323d6d36b408"
format_stamp: "Formatted at 2026-08-11 02:56:03 on dist-test-slave-9gcw"
server_key: "2b41d53087a3d732dd720a4f07a32edb"
server_key_iv: "02bfb822e6b0a669fb1401a1cf91043b"
server_key_version: "encryptionkey@0"
I20260811 02:56:03.082677 307 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786416707705494-12843-0/minicluster-data/master-0/wal/instance:
uuid: "4ec9fb2d07bf4bb38266323d6d36b408"
format_stamp: "Formatted at 2026-08-11 02:56:03 on dist-test-slave-9gcw"
server_key: "2b41d53087a3d732dd720a4f07a32edb"
server_key_iv: "02bfb822e6b0a669fb1401a1cf91043b"
server_key_version: "encryptionkey@0"
I20260811 02:56:03.086687 307 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260811 02:56:03.089221 326 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 02:56:03.090512 307 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260811 02:56:03.090672 307 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786416707705494-12843-0/minicluster-data/master-0/data,/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786416707705494-12843-0/minicluster-data/master-0/wal
uuid: "4ec9fb2d07bf4bb38266323d6d36b408"
format_stamp: "Formatted at 2026-08-11 02:56:03 on dist-test-slave-9gcw"
server_key: "2b41d53087a3d732dd720a4f07a32edb"
server_key_iv: "02bfb822e6b0a669fb1401a1cf91043b"
server_key_version: "encryptionkey@0"
I20260811 02:56:03.090806 307 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786416707705494-12843-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786416707705494-12843-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786416707705494-12843-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 02:56:03.102492 307 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 02:56:03.106170 307 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 02:56:03.106406 307 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 02:56:03.114264 307 rpc_server.cc:307] RPC server started. Bound to: 127.12.138.254:40905
I20260811 02:56:03.114295 382 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.138.254:40905 every 8 connection(s)
I20260811 02:56:03.115296 307 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786416707705494-12843-0/minicluster-data/master-0/data/info.pb
I20260811 02:56:03.115698 12843 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDawEwS/build/debug/bin/kudu as pid 307
I20260811 02:56:03.115878 12843 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786416707705494-12843-0/minicluster-data/master-0/wal/instance
I20260811 02:56:03.116266 12843 external_mini_cluster.cc:1428] Setting key 016bff1aad89fd18f75820652d8904f1
I20260811 02:56:03.118664 384 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 02:56:03.125617 384 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4ec9fb2d07bf4bb38266323d6d36b408: Bootstrap starting.
Aug 11 02:56:03 dist-test-slave-9gcw krb5kdc[32761](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416962, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.12.138.254@KRBTEST.COM
I20260811 02:56:03.128372 384 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4ec9fb2d07bf4bb38266323d6d36b408: Neither blocks nor log segments found. Creating new log.
I20260811 02:56:03.129379 384 log.cc:824] T 00000000000000000000000000000000 P 4ec9fb2d07bf4bb38266323d6d36b408: Log is configured to *not* fsync() on all Append() calls
I20260811 02:56:03.131880 384 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4ec9fb2d07bf4bb38266323d6d36b408: No bootstrap required, opened a new log
I20260811 02:56:03.132795 388 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:56:03.117805 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44742 (local address 127.12.138.254:40905)
0811 02:56:03.118273 (+ 468us) server_negotiation.cc:206] Beginning negotiation
0811 02:56:03.118290 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0811 02:56:03.118331 (+ 41us) server_negotiation.cc:407] Connection header received
0811 02:56:03.119122 (+ 791us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:56:03.119144 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:56:03.119507 (+ 363us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:56:03.119834 (+ 327us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:56:03.120878 (+ 1044us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:56:03.122215 (+ 1337us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:56:03.123433 (+ 1218us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:56:03.123944 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:56:03.124074 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:56:03.127189 (+ 3115us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:56:03.127196 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:56:03.127212 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:56:03.127245 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:56:03.129455 (+ 2210us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:56:03.130013 (+ 558us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:56:03.130018 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:56:03.130022 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:56:03.130137 (+ 115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:56:03.130453 (+ 316us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:56:03.130456 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:56:03.130458 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:56:03.131017 (+ 559us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:56:03.131215 (+ 198us) server_negotiation.cc:1035] Waiting for connection context
0811 02:56:03.131482 (+ 267us) server_negotiation.cc:299] Negotiation successful
0811 02:56:03.131811 (+ 329us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":304,"thread_start_us":164,"threads_started":1}
I20260811 02:56:03.136066 384 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4ec9fb2d07bf4bb38266323d6d36b408 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4ec9fb2d07bf4bb38266323d6d36b408" member_type: VOTER last_known_addr { host: "127.12.138.254" port: 40905 } }
I20260811 02:56:03.136301 384 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4ec9fb2d07bf4bb38266323d6d36b408 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 02:56:03.136338 384 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4ec9fb2d07bf4bb38266323d6d36b408 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4ec9fb2d07bf4bb38266323d6d36b408, State: Initialized, Role: FOLLOWER
I20260811 02:56:03.136698 384 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4ec9fb2d07bf4bb38266323d6d36b408 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4ec9fb2d07bf4bb38266323d6d36b408" member_type: VOTER last_known_addr { host: "127.12.138.254" port: 40905 } }
I20260811 02:56:03.136811 384 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4ec9fb2d07bf4bb38266323d6d36b408 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 02:56:03.136857 384 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4ec9fb2d07bf4bb38266323d6d36b408 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 02:56:03.136927 384 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4ec9fb2d07bf4bb38266323d6d36b408 [term 0 FOLLOWER]: Advancing to term 1
I20260811 02:56:03.137887 384 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4ec9fb2d07bf4bb38266323d6d36b408 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4ec9fb2d07bf4bb38266323d6d36b408" member_type: VOTER last_known_addr { host: "127.12.138.254" port: 40905 } }
I20260811 02:56:03.138185 384 leader_election.cc:302] T 00000000000000000000000000000000 P 4ec9fb2d07bf4bb38266323d6d36b408 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4ec9fb2d07bf4bb38266323d6d36b408; no voters:
I20260811 02:56:03.138456 384 leader_election.cc:288] T 00000000000000000000000000000000 P 4ec9fb2d07bf4bb38266323d6d36b408 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 02:56:03.138618 391 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4ec9fb2d07bf4bb38266323d6d36b408 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 02:56:03.138888 391 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4ec9fb2d07bf4bb38266323d6d36b408 [term 1 LEADER]: Becoming Leader. State: Replica: 4ec9fb2d07bf4bb38266323d6d36b408, State: Running, Role: LEADER
I20260811 02:56:03.139206 391 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4ec9fb2d07bf4bb38266323d6d36b408 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4ec9fb2d07bf4bb38266323d6d36b408" member_type: VOTER last_known_addr { host: "127.12.138.254" port: 40905 } }
I20260811 02:56:03.139783 384 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4ec9fb2d07bf4bb38266323d6d36b408 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 02:56:03.140949 392 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4ec9fb2d07bf4bb38266323d6d36b408 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4ec9fb2d07bf4bb38266323d6d36b408" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4ec9fb2d07bf4bb38266323d6d36b408" member_type: VOTER last_known_addr { host: "127.12.138.254" port: 40905 } } }
I20260811 02:56:03.141083 392 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4ec9fb2d07bf4bb38266323d6d36b408 [sys.catalog]: This master's current role is: LEADER
I20260811 02:56:03.141445 393 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4ec9fb2d07bf4bb38266323d6d36b408 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4ec9fb2d07bf4bb38266323d6d36b408. Latest consensus state: current_term: 1 leader_uuid: "4ec9fb2d07bf4bb38266323d6d36b408" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4ec9fb2d07bf4bb38266323d6d36b408" member_type: VOTER last_known_addr { host: "127.12.138.254" port: 40905 } } }
I20260811 02:56:03.141548 393 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4ec9fb2d07bf4bb38266323d6d36b408 [sys.catalog]: This master's current role is: LEADER
I20260811 02:56:03.142284 400 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 02:56:03.145639 400 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 02:56:03.151017 400 catalog_manager.cc:1347] Generated new cluster ID: 513975e7513a478a9d021f2f7f969c7b
I20260811 02:56:03.151094 400 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 02:56:03.174712 400 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 02:56:03.176287 400 catalog_manager.cc:1504] Loading token signing keys...
I20260811 02:56:03.199432 400 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4ec9fb2d07bf4bb38266323d6d36b408: Generated new TSK 0
I20260811 02:56:03.200325 400 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.12.138.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.138.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.138.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786416707705494-12843-0/krb5kdc/kudu_127.12.138.193.keytab.
Entry for principal kudu/127.12.138.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786416707705494-12843-0/krb5kdc/kudu_127.12.138.193.keytab.
I20260811 02:56:03.240554 12843 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDawEwS/build/debug/bin/kudu
/tmp/dist-test-taskDawEwS/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786416707705494-12843-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786416707705494-12843-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786416707705494-12843-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786416707705494-12843-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.138.193:0
--local_ip_for_outbound_sockets=127.12.138.193
--webserver_interface=127.12.138.193
--webserver_port=0
--tserver_master_addrs=127.12.138.254:40905
--keytab_file=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786416707705494-12843-0/krb5kdc/kudu_127.12.138.193.keytab
--principal=kudu/127.12.138.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.138.212:38789
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786416707705494-12843-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786416707705494-12843-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786416707705494-12843-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 02:56:03.375809 416 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 02:56:03.376122 416 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 02:56:03.376173 416 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 02:56:03.380435 416 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 02:56:03.380515 416 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 02:56:03.380575 416 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.138.193
I20260811 02:56:03.386040 416 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.138.212:38789
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786416707705494-12843-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786416707705494-12843-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786416707705494-12843-0/krb5kdc/kudu_127.12.138.193.keytab
--principal=kudu/127.12.138.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.138.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786416707705494-12843-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.12.138.193
--webserver_port=0
--tserver_master_addrs=127.12.138.254:40905
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.416
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.138.193
--log_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786416707705494-12843-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 02:09:49 UTC on ccycloud.quasar-fsnqqc.root.comops.site
build id 4579570
I20260811 02:56:03.387178 416 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 02:56:03.388244 416 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 02:56:03.398188 423 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:56:03.398113 426 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:56:03.398113 424 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:56:03.398530 416 server_base.cc:1034] running on GCE node
I20260811 02:56:03.399025 416 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 02:56:03.399680 416 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 02:56:03.400890 416 hybrid_clock.cc:648] HybridClock initialized: now 1786416963400860 us; error 50 us; skew 500 ppm
Aug 11 02:56:03 dist-test-slave-9gcw krb5kdc[32761](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416963, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.138.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 02:56:03.403915 416 init.cc:377] Logged in from keytab as kudu/127.12.138.193@KRBTEST.COM (short username kudu)
I20260811 02:56:03.404909 416 webserver.cc:460] Webserver started at http://127.12.138.193:44849/ using document root <none> and password file <none>
I20260811 02:56:03.405561 416 fs_manager.cc:359] Metadata directory not provided
I20260811 02:56:03.405668 416 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 02:56:03.405921 416 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 02:56:03.408042 416 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786416707705494-12843-0/minicluster-data/ts-0/data/instance:
uuid: "279f9395b2d54df2a22f4a7a0b74244b"
format_stamp: "Formatted at 2026-08-11 02:56:03 on dist-test-slave-9gcw"
server_key: "34a608a98b5cdaa184c64e383f842690"
server_key_iv: "83b3ef7538a022383dd4ee9d7aecd07b"
server_key_version: "encryptionkey@0"
I20260811 02:56:03.408635 416 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786416707705494-12843-0/minicluster-data/ts-0/wal/instance:
uuid: "279f9395b2d54df2a22f4a7a0b74244b"
format_stamp: "Formatted at 2026-08-11 02:56:03 on dist-test-slave-9gcw"
server_key: "34a608a98b5cdaa184c64e383f842690"
server_key_iv: "83b3ef7538a022383dd4ee9d7aecd07b"
server_key_version: "encryptionkey@0"
I20260811 02:56:03.412495 416 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260811 02:56:03.415200 434 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 02:56:03.416519 416 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260811 02:56:03.416720 416 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786416707705494-12843-0/minicluster-data/ts-0/data,/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786416707705494-12843-0/minicluster-data/ts-0/wal
uuid: "279f9395b2d54df2a22f4a7a0b74244b"
format_stamp: "Formatted at 2026-08-11 02:56:03 on dist-test-slave-9gcw"
server_key: "34a608a98b5cdaa184c64e383f842690"
server_key_iv: "83b3ef7538a022383dd4ee9d7aecd07b"
server_key_version: "encryptionkey@0"
I20260811 02:56:03.416850 416 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786416707705494-12843-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786416707705494-12843-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786416707705494-12843-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 02:56:03.432662 416 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 02:56:03.435882 416 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 02:56:03.436111 416 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 02:56:03.436759 416 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 02:56:03.437673 416 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 02:56:03.437762 416 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 02:56:03.437839 416 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 02:56:03.437871 416 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 02:56:03.449311 416 rpc_server.cc:307] RPC server started. Bound to: 127.12.138.193:39457
I20260811 02:56:03.449360 558 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.138.193:39457 every 8 connection(s)
I20260811 02:56:03.450369 416 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786416707705494-12843-0/minicluster-data/ts-0/data/info.pb
I20260811 02:56:03.460057 12843 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDawEwS/build/debug/bin/kudu as pid 416
I20260811 02:56:03.460218 12843 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786416707705494-12843-0/minicluster-data/ts-0/wal/instance
I20260811 02:56:03.460604 12843 external_mini_cluster.cc:1428] Setting key 1e8c2283a176f08baeec641215ae0cba
Aug 11 02:56:03 dist-test-slave-9gcw krb5kdc[32761](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416963, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.138.193@KRBTEST.COM for kudu/127.12.138.254@KRBTEST.COM
I20260811 02:56:03.465898 388 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:56:03.452597 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.138.193:60667 (local address 127.12.138.254:40905)
0811 02:56:03.452785 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0811 02:56:03.452791 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 02:56:03.453969 (+ 1178us) server_negotiation.cc:407] Connection header received
0811 02:56:03.455353 (+ 1384us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:56:03.455357 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:56:03.455412 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:56:03.455506 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:56:03.456845 (+ 1339us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:56:03.457505 (+ 660us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:56:03.458828 (+ 1323us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:56:03.459088 (+ 260us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:56:03.459193 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:56:03.462117 (+ 2924us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:56:03.462121 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:56:03.462123 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:56:03.462147 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:56:03.463647 (+ 1500us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:56:03.464208 (+ 561us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:56:03.464214 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:56:03.464216 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:56:03.464298 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:56:03.464656 (+ 358us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:56:03.464661 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:56:03.464663 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:56:03.464861 (+ 198us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:56:03.464973 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0811 02:56:03.465577 (+ 604us) server_negotiation.cc:299] Negotiation successful
0811 02:56:03.465725 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260811 02:56:03.466792 561 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:56:03.453079 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.138.254:40905 (local address 127.12.138.193:60667)
0811 02:56:03.453790 (+ 711us) negotiation.cc:107] Waiting for socket to connect
0811 02:56:03.453839 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0811 02:56:03.455091 (+ 1252us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:56:03.455757 (+ 666us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:56:03.455764 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:56:03.456144 (+ 380us) client_negotiation.cc:189] Negotiated authn=SASL
0811 02:56:03.456656 (+ 512us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:56:03.456680 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:56:03.457660 (+ 980us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:56:03.457663 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:56:03.458664 (+ 1001us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:56:03.458675 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:56:03.459244 (+ 569us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:56:03.459246 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:56:03.459399 (+ 153us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:56:03.460025 (+ 626us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 02:56:03.460049 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0811 02:56:03.461931 (+ 1882us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 02:56:03.463787 (+ 1856us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:56:03.463793 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:56:03.463810 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:56:03.464091 (+ 281us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:56:03.464395 (+ 304us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:56:03.464399 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:56:03.464401 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:56:03.464554 (+ 153us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:56:03.464984 (+ 430us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 02:56:03.464992 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 02:56:03.465332 (+ 340us) client_negotiation.cc:769] Sending connection context
0811 02:56:03.465534 (+ 202us) client_negotiation.cc:240] Negotiation successful
0811 02:56:03.465906 (+ 372us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":410,"thread_start_us":135,"threads_started":1}
I20260811 02:56:03.468240 559 heartbeater.cc:347] Connected to a master server at 127.12.138.254:40905
I20260811 02:56:03.468513 559 heartbeater.cc:464] Registering TS with master...
I20260811 02:56:03.469046 559 heartbeater.cc:511] Master 127.12.138.254:40905 requested a full tablet report, sending...
I20260811 02:56:03.470642 344 ts_manager.cc:194] Registered new tserver with Master: 279f9395b2d54df2a22f4a7a0b74244b (127.12.138.193:39457)
WARNING: no policy specified for kudu/127.12.138.194@KRBTEST.COM; defaulting to no policy
I20260811 02:56:03.471937 344 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.12.138.193@KRBTEST.COM'} at 127.12.138.193:60667
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.138.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.138.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786416707705494-12843-0/krb5kdc/kudu_127.12.138.194.keytab.
Entry for principal kudu/127.12.138.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786416707705494-12843-0/krb5kdc/kudu_127.12.138.194.keytab.
I20260811 02:56:03.492579 12843 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDawEwS/build/debug/bin/kudu
/tmp/dist-test-taskDawEwS/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786416707705494-12843-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786416707705494-12843-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786416707705494-12843-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786416707705494-12843-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.138.194:0
--local_ip_for_outbound_sockets=127.12.138.194
--webserver_interface=127.12.138.194
--webserver_port=0
--tserver_master_addrs=127.12.138.254:40905
--keytab_file=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786416707705494-12843-0/krb5kdc/kudu_127.12.138.194.keytab
--principal=kudu/127.12.138.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.138.212:38789
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786416707705494-12843-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786416707705494-12843-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786416707705494-12843-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 02:56:03.605861 565 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 02:56:03.606164 565 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 02:56:03.606246 565 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 02:56:03.610193 565 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 02:56:03.610282 565 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 02:56:03.610373 565 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.138.194
I20260811 02:56:03.615540 565 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.138.212:38789
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786416707705494-12843-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786416707705494-12843-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786416707705494-12843-0/krb5kdc/kudu_127.12.138.194.keytab
--principal=kudu/127.12.138.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.138.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786416707705494-12843-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.12.138.194
--webserver_port=0
--tserver_master_addrs=127.12.138.254:40905
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.565
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.138.194
--log_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786416707705494-12843-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 02:09:49 UTC on ccycloud.quasar-fsnqqc.root.comops.site
build id 4579570
I20260811 02:56:03.616626 565 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 02:56:03.617678 565 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 02:56:03.627921 571 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:56:03.628055 572 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:56:03.627923 574 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:56:03.628146 565 server_base.cc:1034] running on GCE node
I20260811 02:56:03.628688 565 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 02:56:03.629428 565 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 02:56:03.630602 565 hybrid_clock.cc:648] HybridClock initialized: now 1786416963630568 us; error 46 us; skew 500 ppm
Aug 11 02:56:03 dist-test-slave-9gcw krb5kdc[32761](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416963, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.138.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 02:56:03.633538 565 init.cc:377] Logged in from keytab as kudu/127.12.138.194@KRBTEST.COM (short username kudu)
I20260811 02:56:03.634430 565 webserver.cc:460] Webserver started at http://127.12.138.194:46427/ using document root <none> and password file <none>
I20260811 02:56:03.635010 565 fs_manager.cc:359] Metadata directory not provided
I20260811 02:56:03.635070 565 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 02:56:03.635242 565 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 02:56:03.637394 565 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786416707705494-12843-0/minicluster-data/ts-1/data/instance:
uuid: "59c854d7d7c5440facc10cdcf9626ac5"
format_stamp: "Formatted at 2026-08-11 02:56:03 on dist-test-slave-9gcw"
server_key: "f4f34fabe873b26d8d6213f0b17d8636"
server_key_iv: "987b79268aead07defb5df0406dda337"
server_key_version: "encryptionkey@0"
I20260811 02:56:03.637930 565 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786416707705494-12843-0/minicluster-data/ts-1/wal/instance:
uuid: "59c854d7d7c5440facc10cdcf9626ac5"
format_stamp: "Formatted at 2026-08-11 02:56:03 on dist-test-slave-9gcw"
server_key: "f4f34fabe873b26d8d6213f0b17d8636"
server_key_iv: "987b79268aead07defb5df0406dda337"
server_key_version: "encryptionkey@0"
I20260811 02:56:03.642091 565 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260811 02:56:03.645099 581 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 02:56:03.646517 565 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260811 02:56:03.646617 565 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786416707705494-12843-0/minicluster-data/ts-1/data,/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786416707705494-12843-0/minicluster-data/ts-1/wal
uuid: "59c854d7d7c5440facc10cdcf9626ac5"
format_stamp: "Formatted at 2026-08-11 02:56:03 on dist-test-slave-9gcw"
server_key: "f4f34fabe873b26d8d6213f0b17d8636"
server_key_iv: "987b79268aead07defb5df0406dda337"
server_key_version: "encryptionkey@0"
I20260811 02:56:03.646705 565 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786416707705494-12843-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786416707705494-12843-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786416707705494-12843-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 02:56:03.666481 565 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 02:56:03.670156 565 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 02:56:03.670324 565 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 02:56:03.670915 565 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 02:56:03.671778 565 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 02:56:03.671831 565 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 02:56:03.671877 565 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 02:56:03.671895 565 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 02:56:03.682565 565 rpc_server.cc:307] RPC server started. Bound to: 127.12.138.194:35029
I20260811 02:56:03.682585 705 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.138.194:35029 every 8 connection(s)
I20260811 02:56:03.683533 565 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786416707705494-12843-0/minicluster-data/ts-1/data/info.pb
I20260811 02:56:03.690059 12843 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDawEwS/build/debug/bin/kudu as pid 565
I20260811 02:56:03.690178 12843 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786416707705494-12843-0/minicluster-data/ts-1/wal/instance
I20260811 02:56:03.690526 12843 external_mini_cluster.cc:1428] Setting key ded96581c2599847a74839da9b57ac1c
Aug 11 02:56:03 dist-test-slave-9gcw krb5kdc[32761](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416963, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.138.194@KRBTEST.COM for kudu/127.12.138.254@KRBTEST.COM
I20260811 02:56:03.700121 388 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:56:03.685876 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.138.194:46399 (local address 127.12.138.254:40905)
0811 02:56:03.686070 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0811 02:56:03.686076 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 02:56:03.687034 (+ 958us) server_negotiation.cc:407] Connection header received
0811 02:56:03.688415 (+ 1381us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:56:03.688418 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:56:03.688466 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:56:03.688546 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:56:03.689971 (+ 1425us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:56:03.690730 (+ 759us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:56:03.692214 (+ 1484us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:56:03.692466 (+ 252us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:56:03.692597 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:56:03.695885 (+ 3288us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:56:03.695888 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:56:03.695890 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:56:03.695916 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:56:03.697567 (+ 1651us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:56:03.698305 (+ 738us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:56:03.698308 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:56:03.698309 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:56:03.698391 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:56:03.698847 (+ 456us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:56:03.698850 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:56:03.698851 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:56:03.699035 (+ 184us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:56:03.699147 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0811 02:56:03.699890 (+ 743us) server_negotiation.cc:299] Negotiation successful
0811 02:56:03.699997 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260811 02:56:03.701453 708 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:56:03.686276 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.138.254:40905 (local address 127.12.138.194:46399)
0811 02:56:03.686857 (+ 581us) negotiation.cc:107] Waiting for socket to connect
0811 02:56:03.686906 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0811 02:56:03.688171 (+ 1265us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:56:03.688792 (+ 621us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:56:03.688799 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:56:03.689284 (+ 485us) client_negotiation.cc:189] Negotiated authn=SASL
0811 02:56:03.689802 (+ 518us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:56:03.689817 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:56:03.690872 (+ 1055us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:56:03.690881 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:56:03.692008 (+ 1127us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:56:03.692022 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:56:03.692644 (+ 622us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:56:03.692649 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:56:03.692832 (+ 183us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:56:03.693548 (+ 716us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 02:56:03.693566 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0811 02:56:03.695475 (+ 1909us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 02:56:03.697737 (+ 2262us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:56:03.697745 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:56:03.697765 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:56:03.698193 (+ 428us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:56:03.698499 (+ 306us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:56:03.698505 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:56:03.698510 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:56:03.698725 (+ 215us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:56:03.699162 (+ 437us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 02:56:03.699172 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 02:56:03.699586 (+ 414us) client_negotiation.cc:769] Sending connection context
0811 02:56:03.699857 (+ 271us) client_negotiation.cc:240] Negotiation successful
0811 02:56:03.700258 (+ 401us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":375,"thread_start_us":126,"threads_started":1}
I20260811 02:56:03.702716 706 heartbeater.cc:347] Connected to a master server at 127.12.138.254:40905
WARNING: no policy specified for kudu/127.12.138.195@KRBTEST.COM; defaulting to no policy
I20260811 02:56:03.703024 706 heartbeater.cc:464] Registering TS with master...
I20260811 02:56:03.703613 706 heartbeater.cc:511] Master 127.12.138.254:40905 requested a full tablet report, sending...
I20260811 02:56:03.704690 344 ts_manager.cc:194] Registered new tserver with Master: 59c854d7d7c5440facc10cdcf9626ac5 (127.12.138.194:35029)
I20260811 02:56:03.705392 344 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.12.138.194@KRBTEST.COM'} at 127.12.138.194:46399
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.138.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.138.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786416707705494-12843-0/krb5kdc/kudu_127.12.138.195.keytab.
Entry for principal kudu/127.12.138.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786416707705494-12843-0/krb5kdc/kudu_127.12.138.195.keytab.
I20260811 02:56:03.724148 12843 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDawEwS/build/debug/bin/kudu
/tmp/dist-test-taskDawEwS/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786416707705494-12843-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786416707705494-12843-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786416707705494-12843-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786416707705494-12843-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.138.195:0
--local_ip_for_outbound_sockets=127.12.138.195
--webserver_interface=127.12.138.195
--webserver_port=0
--tserver_master_addrs=127.12.138.254:40905
--keytab_file=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786416707705494-12843-0/krb5kdc/kudu_127.12.138.195.keytab
--principal=kudu/127.12.138.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.138.212:38789
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786416707705494-12843-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786416707705494-12843-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786416707705494-12843-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 02:56:03.841727 712 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 02:56:03.841991 712 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 02:56:03.842034 712 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 02:56:03.846004 712 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 02:56:03.846060 712 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 02:56:03.846164 712 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.138.195
I20260811 02:56:03.851627 712 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.138.212:38789
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786416707705494-12843-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786416707705494-12843-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786416707705494-12843-0/krb5kdc/kudu_127.12.138.195.keytab
--principal=kudu/127.12.138.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.138.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786416707705494-12843-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.12.138.195
--webserver_port=0
--tserver_master_addrs=127.12.138.254:40905
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.712
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.138.195
--log_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786416707705494-12843-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 02:09:49 UTC on ccycloud.quasar-fsnqqc.root.comops.site
build id 4579570
I20260811 02:56:03.852689 712 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 02:56:03.853776 712 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 02:56:03.863690 717 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:56:03.863691 718 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:56:03.863754 720 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:56:03.864023 712 server_base.cc:1034] running on GCE node
I20260811 02:56:03.864408 712 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 02:56:03.865015 712 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 02:56:03.866178 712 hybrid_clock.cc:648] HybridClock initialized: now 1786416963866143 us; error 52 us; skew 500 ppm
Aug 11 02:56:03 dist-test-slave-9gcw krb5kdc[32761](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416963, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.138.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 02:56:03.869189 712 init.cc:377] Logged in from keytab as kudu/127.12.138.195@KRBTEST.COM (short username kudu)
I20260811 02:56:03.870064 712 webserver.cc:460] Webserver started at http://127.12.138.195:45169/ using document root <none> and password file <none>
I20260811 02:56:03.870601 712 fs_manager.cc:359] Metadata directory not provided
I20260811 02:56:03.870672 712 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 02:56:03.870846 712 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 02:56:03.872927 712 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786416707705494-12843-0/minicluster-data/ts-2/data/instance:
uuid: "44daa1d55934407699906055b6e717b7"
format_stamp: "Formatted at 2026-08-11 02:56:03 on dist-test-slave-9gcw"
server_key: "6d5827388eb8ca727eccc4b97dacc09f"
server_key_iv: "a606ee84f116655378bc06b115021b5b"
server_key_version: "encryptionkey@0"
I20260811 02:56:03.873768 712 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786416707705494-12843-0/minicluster-data/ts-2/wal/instance:
uuid: "44daa1d55934407699906055b6e717b7"
format_stamp: "Formatted at 2026-08-11 02:56:03 on dist-test-slave-9gcw"
server_key: "6d5827388eb8ca727eccc4b97dacc09f"
server_key_iv: "a606ee84f116655378bc06b115021b5b"
server_key_version: "encryptionkey@0"
I20260811 02:56:03.878044 712 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260811 02:56:03.880578 728 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 02:56:03.881963 712 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260811 02:56:03.882113 712 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786416707705494-12843-0/minicluster-data/ts-2/data,/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786416707705494-12843-0/minicluster-data/ts-2/wal
uuid: "44daa1d55934407699906055b6e717b7"
format_stamp: "Formatted at 2026-08-11 02:56:03 on dist-test-slave-9gcw"
server_key: "6d5827388eb8ca727eccc4b97dacc09f"
server_key_iv: "a606ee84f116655378bc06b115021b5b"
server_key_version: "encryptionkey@0"
I20260811 02:56:03.882236 712 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786416707705494-12843-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786416707705494-12843-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786416707705494-12843-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 02:56:03.909514 712 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 02:56:03.913261 712 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 02:56:03.913499 712 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 02:56:03.914104 712 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 02:56:03.914933 712 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 02:56:03.914981 712 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 02:56:03.915021 712 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 02:56:03.915045 712 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 02:56:03.924410 712 rpc_server.cc:307] RPC server started. Bound to: 127.12.138.195:46619
I20260811 02:56:03.924439 858 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.138.195:46619 every 8 connection(s)
I20260811 02:56:03.925419 712 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786416707705494-12843-0/minicluster-data/ts-2/data/info.pb
I20260811 02:56:03.932543 12843 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDawEwS/build/debug/bin/kudu as pid 712
I20260811 02:56:03.932664 12843 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786416707705494-12843-0/minicluster-data/ts-2/wal/instance
I20260811 02:56:03.932999 12843 external_mini_cluster.cc:1428] Setting key 47720d12a492e05854e6ee935786eab5
Aug 11 02:56:03 dist-test-slave-9gcw krb5kdc[32761](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416963, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.138.195@KRBTEST.COM for kudu/127.12.138.254@KRBTEST.COM
I20260811 02:56:03.940205 388 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:56:03.927399 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.138.195:37377 (local address 127.12.138.254:40905)
0811 02:56:03.927547 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0811 02:56:03.927550 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 02:56:03.928296 (+ 746us) server_negotiation.cc:407] Connection header received
0811 02:56:03.929753 (+ 1457us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:56:03.929757 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:56:03.929814 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:56:03.929928 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:56:03.931099 (+ 1171us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:56:03.931638 (+ 539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:56:03.932908 (+ 1270us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:56:03.933264 (+ 356us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:56:03.933380 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:56:03.936216 (+ 2836us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:56:03.936219 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:56:03.936221 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:56:03.936248 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:56:03.938078 (+ 1830us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:56:03.938606 (+ 528us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:56:03.938609 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:56:03.938610 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:56:03.938717 (+ 107us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:56:03.939023 (+ 306us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:56:03.939025 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:56:03.939027 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:56:03.939239 (+ 212us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:56:03.939324 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0811 02:56:03.939922 (+ 598us) server_negotiation.cc:299] Negotiation successful
0811 02:56:03.940054 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260811 02:56:03.941284 862 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:56:03.927647 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.138.254:40905 (local address 127.12.138.195:37377)
0811 02:56:03.928140 (+ 493us) negotiation.cc:107] Waiting for socket to connect
0811 02:56:03.928182 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0811 02:56:03.929526 (+ 1344us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:56:03.930103 (+ 577us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:56:03.930109 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:56:03.930466 (+ 357us) client_negotiation.cc:189] Negotiated authn=SASL
0811 02:56:03.930950 (+ 484us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:56:03.930963 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:56:03.931780 (+ 817us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:56:03.931783 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:56:03.932788 (+ 1005us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:56:03.932799 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:56:03.933422 (+ 623us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:56:03.933427 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:56:03.933610 (+ 183us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:56:03.934318 (+ 708us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 02:56:03.934336 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0811 02:56:03.936048 (+ 1712us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 02:56:03.938232 (+ 2184us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:56:03.938237 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:56:03.938249 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:56:03.938505 (+ 256us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:56:03.938809 (+ 304us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:56:03.938811 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:56:03.938812 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:56:03.938940 (+ 128us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:56:03.939369 (+ 429us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 02:56:03.939374 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 02:56:03.939618 (+ 244us) client_negotiation.cc:769] Sending connection context
0811 02:56:03.939818 (+ 200us) client_negotiation.cc:240] Negotiation successful
0811 02:56:03.940176 (+ 358us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":308,"thread_start_us":134,"threads_started":1}
I20260811 02:56:03.942565 859 heartbeater.cc:347] Connected to a master server at 127.12.138.254:40905
I20260811 02:56:03.942835 859 heartbeater.cc:464] Registering TS with master...
I20260811 02:56:03.943341 859 heartbeater.cc:511] Master 127.12.138.254:40905 requested a full tablet report, sending...
I20260811 02:56:03.944460 344 ts_manager.cc:194] Registered new tserver with Master: 44daa1d55934407699906055b6e717b7 (127.12.138.195:46619)
I20260811 02:56:03.945044 344 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.12.138.195@KRBTEST.COM'} at 127.12.138.195:37377
I20260811 02:56:03.947703 12843 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 02:56:03.958043 388 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:56:03.950051 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44750 (local address 127.12.138.254:40905)
0811 02:56:03.950199 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0811 02:56:03.950204 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 02:56:03.950420 (+ 216us) server_negotiation.cc:407] Connection header received
0811 02:56:03.950548 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:56:03.950554 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:56:03.950609 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:56:03.950679 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:56:03.951499 (+ 820us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:56:03.952059 (+ 560us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:56:03.952990 (+ 931us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:56:03.953299 (+ 309us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:56:03.953380 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:56:03.954885 (+ 1505us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:56:03.954888 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:56:03.954890 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:56:03.954912 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:56:03.956338 (+ 1426us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:56:03.956753 (+ 415us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:56:03.956756 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:56:03.956757 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:56:03.956833 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:56:03.957068 (+ 235us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:56:03.957070 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:56:03.957071 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:56:03.957357 (+ 286us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:56:03.957469 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0811 02:56:03.957759 (+ 290us) server_negotiation.cc:299] Negotiation successful
0811 02:56:03.957850 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260811 02:56:03.961095 344 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44750:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 02:56:03.964257 344 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 02:56:03.979669 873 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:56:03.973583 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.138.194:35029 (local address 127.0.0.1:53802)
0811 02:56:03.974893 (+ 1310us) negotiation.cc:107] Waiting for socket to connect
0811 02:56:03.974905 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 02:56:03.974980 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:56:03.975492 (+ 512us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:56:03.975495 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:56:03.975514 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 02:56:03.975622 (+ 108us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:56:03.975631 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:56:03.976946 (+ 1315us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:56:03.976949 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:56:03.978077 (+ 1128us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:56:03.978087 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:56:03.979091 (+ 1004us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:56:03.979095 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:56:03.979211 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:56:03.979259 (+ 48us) client_negotiation.cc:769] Sending connection context
0811 02:56:03.979381 (+ 122us) client_negotiation.cc:240] Negotiation successful
0811 02:56:03.979501 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1242,"mutex_wait_us":21,"thread_start_us":110,"threads_started":1}
I20260811 02:56:03.979694 880 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:56:03.974223 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53802 (local address 127.12.138.194:35029)
0811 02:56:03.975014 (+ 791us) server_negotiation.cc:206] Beginning negotiation
0811 02:56:03.975024 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 02:56:03.975047 (+ 23us) server_negotiation.cc:407] Connection header received
0811 02:56:03.975121 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:56:03.975128 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:56:03.975318 (+ 190us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:56:03.975516 (+ 198us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 02:56:03.975753 (+ 237us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:56:03.976762 (+ 1009us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:56:03.978206 (+ 1444us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:56:03.978957 (+ 751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:56:03.979085 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:56:03.979221 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0811 02:56:03.979433 (+ 212us) server_negotiation.cc:299] Negotiation successful
0811 02:56:03.979535 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":694,"thread_start_us":119,"threads_started":1}
I20260811 02:56:03.980161 876 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:56:03.973404 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.138.193:39457 (local address 127.0.0.1:38500)
0811 02:56:03.974509 (+ 1105us) negotiation.cc:107] Waiting for socket to connect
0811 02:56:03.974560 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0811 02:56:03.974692 (+ 132us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:56:03.975244 (+ 552us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:56:03.975251 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:56:03.975274 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 02:56:03.975445 (+ 171us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:56:03.975453 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:56:03.976668 (+ 1215us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:56:03.976673 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:56:03.978449 (+ 1776us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:56:03.978460 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:56:03.979713 (+ 1253us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:56:03.979719 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:56:03.979827 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:56:03.979856 (+ 29us) client_negotiation.cc:769] Sending connection context
0811 02:56:03.979943 (+ 87us) client_negotiation.cc:240] Negotiation successful
0811 02:56:03.980016 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":918,"thread_start_us":87,"threads_started":1}
I20260811 02:56:03.980702 879 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:56:03.974218 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58296 (local address 127.12.138.195:46619)
0811 02:56:03.974649 (+ 431us) server_negotiation.cc:206] Beginning negotiation
0811 02:56:03.974666 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0811 02:56:03.975354 (+ 688us) server_negotiation.cc:407] Connection header received
0811 02:56:03.975438 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:56:03.975443 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:56:03.975655 (+ 212us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:56:03.975805 (+ 150us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 02:56:03.976195 (+ 390us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:56:03.977307 (+ 1112us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:56:03.978718 (+ 1411us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:56:03.979475 (+ 757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:56:03.979587 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:56:03.979719 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0811 02:56:03.980459 (+ 740us) server_negotiation.cc:299] Negotiation successful
0811 02:56:03.980565 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":330,"thread_start_us":112,"threads_started":1}
I20260811 02:56:03.980814 875 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:56:03.973566 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38500 (local address 127.12.138.193:39457)
0811 02:56:03.974658 (+ 1092us) server_negotiation.cc:206] Beginning negotiation
0811 02:56:03.974665 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 02:56:03.974690 (+ 25us) server_negotiation.cc:407] Connection header received
0811 02:56:03.974841 (+ 151us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:56:03.974847 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:56:03.975068 (+ 221us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:56:03.975229 (+ 161us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 02:56:03.975589 (+ 360us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:56:03.976545 (+ 956us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:56:03.978584 (+ 2039us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:56:03.979410 (+ 826us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:56:03.979528 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:56:03.979641 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0811 02:56:03.980551 (+ 910us) server_negotiation.cc:299] Negotiation successful
0811 02:56:03.980643 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":988,"thread_start_us":92,"threads_started":1}
I20260811 02:56:03.981308 877 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:56:03.974026 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.138.195:46619 (local address 127.0.0.1:58296)
0811 02:56:03.975129 (+ 1103us) negotiation.cc:107] Waiting for socket to connect
0811 02:56:03.975141 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 02:56:03.975244 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:56:03.975947 (+ 703us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:56:03.975950 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:56:03.975967 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 02:56:03.976063 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:56:03.976068 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:56:03.977449 (+ 1381us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:56:03.977452 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:56:03.978604 (+ 1152us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:56:03.978612 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:56:03.979993 (+ 1381us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:56:03.979998 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:56:03.980131 (+ 133us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:56:03.980156 (+ 25us) client_negotiation.cc:769] Sending connection context
0811 02:56:03.980237 (+ 81us) client_negotiation.cc:240] Negotiation successful
0811 02:56:03.981171 (+ 934us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1060,"mutex_wait_us":3,"thread_start_us":101,"threads_started":1}
I20260811 02:56:03.983457 485 tablet_service.cc:1467] Processing CreateTablet for tablet 4595d2f8cd984c96ae007da15d22bd49 (DEFAULT_TABLE table=test-table [id=6a9790ff9aab4d52887c2cb5ddcc019c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 02:56:03.983275 636 tablet_service.cc:1467] Processing CreateTablet for tablet 4595d2f8cd984c96ae007da15d22bd49 (DEFAULT_TABLE table=test-table [id=6a9790ff9aab4d52887c2cb5ddcc019c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 02:56:03.983530 783 tablet_service.cc:1467] Processing CreateTablet for tablet 4595d2f8cd984c96ae007da15d22bd49 (DEFAULT_TABLE table=test-table [id=6a9790ff9aab4d52887c2cb5ddcc019c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 02:56:03.984462 636 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4595d2f8cd984c96ae007da15d22bd49. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 02:56:03.984462 485 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4595d2f8cd984c96ae007da15d22bd49. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 02:56:03.984462 783 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4595d2f8cd984c96ae007da15d22bd49. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 02:56:03.991207 881 tablet_bootstrap.cc:492] T 4595d2f8cd984c96ae007da15d22bd49 P 44daa1d55934407699906055b6e717b7: Bootstrap starting.
I20260811 02:56:03.992425 884 tablet_bootstrap.cc:492] T 4595d2f8cd984c96ae007da15d22bd49 P 59c854d7d7c5440facc10cdcf9626ac5: Bootstrap starting.
I20260811 02:56:03.993404 881 tablet_bootstrap.cc:654] T 4595d2f8cd984c96ae007da15d22bd49 P 44daa1d55934407699906055b6e717b7: Neither blocks nor log segments found. Creating new log.
I20260811 02:56:03.993479 883 tablet_bootstrap.cc:492] T 4595d2f8cd984c96ae007da15d22bd49 P 279f9395b2d54df2a22f4a7a0b74244b: Bootstrap starting.
I20260811 02:56:03.994481 884 tablet_bootstrap.cc:654] T 4595d2f8cd984c96ae007da15d22bd49 P 59c854d7d7c5440facc10cdcf9626ac5: Neither blocks nor log segments found. Creating new log.
I20260811 02:56:03.994650 881 log.cc:824] T 4595d2f8cd984c96ae007da15d22bd49 P 44daa1d55934407699906055b6e717b7: Log is configured to *not* fsync() on all Append() calls
I20260811 02:56:03.996122 883 tablet_bootstrap.cc:654] T 4595d2f8cd984c96ae007da15d22bd49 P 279f9395b2d54df2a22f4a7a0b74244b: Neither blocks nor log segments found. Creating new log.
I20260811 02:56:03.996167 884 log.cc:824] T 4595d2f8cd984c96ae007da15d22bd49 P 59c854d7d7c5440facc10cdcf9626ac5: Log is configured to *not* fsync() on all Append() calls
I20260811 02:56:03.996771 881 tablet_bootstrap.cc:492] T 4595d2f8cd984c96ae007da15d22bd49 P 44daa1d55934407699906055b6e717b7: No bootstrap required, opened a new log
I20260811 02:56:03.996982 881 ts_tablet_manager.cc:1397] T 4595d2f8cd984c96ae007da15d22bd49 P 44daa1d55934407699906055b6e717b7: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260811 02:56:03.996958 883 log.cc:824] T 4595d2f8cd984c96ae007da15d22bd49 P 279f9395b2d54df2a22f4a7a0b74244b: Log is configured to *not* fsync() on all Append() calls
I20260811 02:56:03.998158 884 tablet_bootstrap.cc:492] T 4595d2f8cd984c96ae007da15d22bd49 P 59c854d7d7c5440facc10cdcf9626ac5: No bootstrap required, opened a new log
I20260811 02:56:03.998342 884 ts_tablet_manager.cc:1397] T 4595d2f8cd984c96ae007da15d22bd49 P 59c854d7d7c5440facc10cdcf9626ac5: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260811 02:56:03.998899 883 tablet_bootstrap.cc:492] T 4595d2f8cd984c96ae007da15d22bd49 P 279f9395b2d54df2a22f4a7a0b74244b: No bootstrap required, opened a new log
I20260811 02:56:03.999078 883 ts_tablet_manager.cc:1397] T 4595d2f8cd984c96ae007da15d22bd49 P 279f9395b2d54df2a22f4a7a0b74244b: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260811 02:56:04.000754 881 raft_consensus.cc:348] T 4595d2f8cd984c96ae007da15d22bd49 P 44daa1d55934407699906055b6e717b7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "279f9395b2d54df2a22f4a7a0b74244b" member_type: VOTER last_known_addr { host: "127.12.138.193" port: 39457 } } peers { permanent_uuid: "59c854d7d7c5440facc10cdcf9626ac5" member_type: VOTER last_known_addr { host: "127.12.138.194" port: 35029 } } peers { permanent_uuid: "44daa1d55934407699906055b6e717b7" member_type: VOTER last_known_addr { host: "127.12.138.195" port: 46619 } }
I20260811 02:56:04.000991 881 raft_consensus.cc:374] T 4595d2f8cd984c96ae007da15d22bd49 P 44daa1d55934407699906055b6e717b7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 02:56:04.001066 881 raft_consensus.cc:729] T 4595d2f8cd984c96ae007da15d22bd49 P 44daa1d55934407699906055b6e717b7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 44daa1d55934407699906055b6e717b7, State: Initialized, Role: FOLLOWER
I20260811 02:56:04.001556 881 consensus_queue.cc:260] T 4595d2f8cd984c96ae007da15d22bd49 P 44daa1d55934407699906055b6e717b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "279f9395b2d54df2a22f4a7a0b74244b" member_type: VOTER last_known_addr { host: "127.12.138.193" port: 39457 } } peers { permanent_uuid: "59c854d7d7c5440facc10cdcf9626ac5" member_type: VOTER last_known_addr { host: "127.12.138.194" port: 35029 } } peers { permanent_uuid: "44daa1d55934407699906055b6e717b7" member_type: VOTER last_known_addr { host: "127.12.138.195" port: 46619 } }
I20260811 02:56:04.002592 881 ts_tablet_manager.cc:1428] T 4595d2f8cd984c96ae007da15d22bd49 P 44daa1d55934407699906055b6e717b7: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
I20260811 02:56:04.002475 884 raft_consensus.cc:348] T 4595d2f8cd984c96ae007da15d22bd49 P 59c854d7d7c5440facc10cdcf9626ac5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "279f9395b2d54df2a22f4a7a0b74244b" member_type: VOTER last_known_addr { host: "127.12.138.193" port: 39457 } } peers { permanent_uuid: "59c854d7d7c5440facc10cdcf9626ac5" member_type: VOTER last_known_addr { host: "127.12.138.194" port: 35029 } } peers { permanent_uuid: "44daa1d55934407699906055b6e717b7" member_type: VOTER last_known_addr { host: "127.12.138.195" port: 46619 } }
I20260811 02:56:04.002823 884 raft_consensus.cc:374] T 4595d2f8cd984c96ae007da15d22bd49 P 59c854d7d7c5440facc10cdcf9626ac5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 02:56:04.002920 884 raft_consensus.cc:729] T 4595d2f8cd984c96ae007da15d22bd49 P 59c854d7d7c5440facc10cdcf9626ac5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 59c854d7d7c5440facc10cdcf9626ac5, State: Initialized, Role: FOLLOWER
I20260811 02:56:04.003016 859 heartbeater.cc:503] Master 127.12.138.254:40905 was elected leader, sending a full tablet report...
I20260811 02:56:04.003363 884 consensus_queue.cc:260] T 4595d2f8cd984c96ae007da15d22bd49 P 59c854d7d7c5440facc10cdcf9626ac5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "279f9395b2d54df2a22f4a7a0b74244b" member_type: VOTER last_known_addr { host: "127.12.138.193" port: 39457 } } peers { permanent_uuid: "59c854d7d7c5440facc10cdcf9626ac5" member_type: VOTER last_known_addr { host: "127.12.138.194" port: 35029 } } peers { permanent_uuid: "44daa1d55934407699906055b6e717b7" member_type: VOTER last_known_addr { host: "127.12.138.195" port: 46619 } }
I20260811 02:56:04.003432 883 raft_consensus.cc:348] T 4595d2f8cd984c96ae007da15d22bd49 P 279f9395b2d54df2a22f4a7a0b74244b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "279f9395b2d54df2a22f4a7a0b74244b" member_type: VOTER last_known_addr { host: "127.12.138.193" port: 39457 } } peers { permanent_uuid: "59c854d7d7c5440facc10cdcf9626ac5" member_type: VOTER last_known_addr { host: "127.12.138.194" port: 35029 } } peers { permanent_uuid: "44daa1d55934407699906055b6e717b7" member_type: VOTER last_known_addr { host: "127.12.138.195" port: 46619 } }
I20260811 02:56:04.003698 883 raft_consensus.cc:374] T 4595d2f8cd984c96ae007da15d22bd49 P 279f9395b2d54df2a22f4a7a0b74244b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 02:56:04.003767 883 raft_consensus.cc:729] T 4595d2f8cd984c96ae007da15d22bd49 P 279f9395b2d54df2a22f4a7a0b74244b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 279f9395b2d54df2a22f4a7a0b74244b, State: Initialized, Role: FOLLOWER
I20260811 02:56:04.004165 883 consensus_queue.cc:260] T 4595d2f8cd984c96ae007da15d22bd49 P 279f9395b2d54df2a22f4a7a0b74244b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "279f9395b2d54df2a22f4a7a0b74244b" member_type: VOTER last_known_addr { host: "127.12.138.193" port: 39457 } } peers { permanent_uuid: "59c854d7d7c5440facc10cdcf9626ac5" member_type: VOTER last_known_addr { host: "127.12.138.194" port: 35029 } } peers { permanent_uuid: "44daa1d55934407699906055b6e717b7" member_type: VOTER last_known_addr { host: "127.12.138.195" port: 46619 } }
I20260811 02:56:04.004372 884 ts_tablet_manager.cc:1428] T 4595d2f8cd984c96ae007da15d22bd49 P 59c854d7d7c5440facc10cdcf9626ac5: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260811 02:56:04.004446 706 heartbeater.cc:503] Master 127.12.138.254:40905 was elected leader, sending a full tablet report...
I20260811 02:56:04.004909 559 heartbeater.cc:503] Master 127.12.138.254:40905 was elected leader, sending a full tablet report...
I20260811 02:56:04.005196 883 ts_tablet_manager.cc:1428] T 4595d2f8cd984c96ae007da15d22bd49 P 279f9395b2d54df2a22f4a7a0b74244b: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
W20260811 02:56:04.176427 861 tablet.cc:2369] T 4595d2f8cd984c96ae007da15d22bd49 P 44daa1d55934407699906055b6e717b7: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 02:56:04.184967 707 tablet.cc:2369] T 4595d2f8cd984c96ae007da15d22bd49 P 59c854d7d7c5440facc10cdcf9626ac5: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 02:56:04.201924 560 tablet.cc:2369] T 4595d2f8cd984c96ae007da15d22bd49 P 279f9395b2d54df2a22f4a7a0b74244b: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 02:56:04.257367 890 raft_consensus.cc:482] T 4595d2f8cd984c96ae007da15d22bd49 P 279f9395b2d54df2a22f4a7a0b74244b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 02:56:04.257596 890 raft_consensus.cc:504] T 4595d2f8cd984c96ae007da15d22bd49 P 279f9395b2d54df2a22f4a7a0b74244b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "279f9395b2d54df2a22f4a7a0b74244b" member_type: VOTER last_known_addr { host: "127.12.138.193" port: 39457 } } peers { permanent_uuid: "59c854d7d7c5440facc10cdcf9626ac5" member_type: VOTER last_known_addr { host: "127.12.138.194" port: 35029 } } peers { permanent_uuid: "44daa1d55934407699906055b6e717b7" member_type: VOTER last_known_addr { host: "127.12.138.195" port: 46619 } }
I20260811 02:56:04.258842 890 leader_election.cc:288] T 4595d2f8cd984c96ae007da15d22bd49 P 279f9395b2d54df2a22f4a7a0b74244b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59c854d7d7c5440facc10cdcf9626ac5 (127.12.138.194:35029), 44daa1d55934407699906055b6e717b7 (127.12.138.195:46619)
I20260811 02:56:04.263166 879 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:56:04.259269 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.138.193:53529 (local address 127.12.138.195:46619)
0811 02:56:04.259495 (+ 226us) server_negotiation.cc:206] Beginning negotiation
0811 02:56:04.259502 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 02:56:04.259713 (+ 211us) server_negotiation.cc:407] Connection header received
0811 02:56:04.259847 (+ 134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:56:04.259851 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:56:04.259905 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:56:04.259998 (+ 93us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 02:56:04.260292 (+ 294us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:56:04.260878 (+ 586us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:56:04.262114 (+ 1236us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:56:04.262549 (+ 435us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:56:04.262648 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:56:04.262733 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0811 02:56:04.262930 (+ 197us) server_negotiation.cc:299] Negotiation successful
0811 02:56:04.263014 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":108}
I20260811 02:56:04.263199 880 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:56:04.259204 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.138.193:43629 (local address 127.12.138.194:35029)
0811 02:56:04.259434 (+ 230us) server_negotiation.cc:206] Beginning negotiation
0811 02:56:04.259441 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 02:56:04.259506 (+ 65us) server_negotiation.cc:407] Connection header received
0811 02:56:04.259698 (+ 192us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:56:04.259709 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:56:04.259769 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:56:04.259855 (+ 86us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 02:56:04.260235 (+ 380us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:56:04.260808 (+ 573us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:56:04.262042 (+ 1234us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:56:04.262568 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:56:04.262787 (+ 219us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:56:04.262835 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0811 02:56:04.263016 (+ 181us) server_negotiation.cc:299] Negotiation successful
0811 02:56:04.263095 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":136}
I20260811 02:56:04.263216 892 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:56:04.259167 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.138.195:46619 (local address 127.12.138.193:53529)
0811 02:56:04.259586 (+ 419us) negotiation.cc:107] Waiting for socket to connect
0811 02:56:04.259597 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0811 02:56:04.259722 (+ 125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:56:04.260029 (+ 307us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:56:04.260031 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:56:04.260046 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 02:56:04.260179 (+ 133us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:56:04.260188 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:56:04.261001 (+ 813us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:56:04.261003 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:56:04.261935 (+ 932us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:56:04.261946 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:56:04.262682 (+ 736us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:56:04.262689 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:56:04.262789 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:56:04.262814 (+ 25us) client_negotiation.cc:769] Sending connection context
0811 02:56:04.262899 (+ 85us) client_negotiation.cc:240] Negotiation successful
0811 02:56:04.263006 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":341,"mutex_wait_us":1,"thread_start_us":109,"threads_started":1}
I20260811 02:56:04.263410 891 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:56:04.259078 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.138.194:35029 (local address 127.12.138.193:43629)
0811 02:56:04.259415 (+ 337us) negotiation.cc:107] Waiting for socket to connect
0811 02:56:04.259430 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 02:56:04.259570 (+ 140us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:56:04.259878 (+ 308us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:56:04.259880 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:56:04.259898 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 02:56:04.260068 (+ 170us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:56:04.260078 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:56:04.260929 (+ 851us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:56:04.260933 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:56:04.261893 (+ 960us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:56:04.261903 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:56:04.262744 (+ 841us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:56:04.262750 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:56:04.262842 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:56:04.262860 (+ 18us) client_negotiation.cc:769] Sending connection context
0811 02:56:04.262970 (+ 110us) client_negotiation.cc:240] Negotiation successful
0811 02:56:04.263033 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":257,"mutex_wait_us":21,"thread_start_us":145,"threads_started":1}
I20260811 02:56:04.263768 889 raft_consensus.cc:482] T 4595d2f8cd984c96ae007da15d22bd49 P 59c854d7d7c5440facc10cdcf9626ac5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 02:56:04.263967 889 raft_consensus.cc:504] T 4595d2f8cd984c96ae007da15d22bd49 P 59c854d7d7c5440facc10cdcf9626ac5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "279f9395b2d54df2a22f4a7a0b74244b" member_type: VOTER last_known_addr { host: "127.12.138.193" port: 39457 } } peers { permanent_uuid: "59c854d7d7c5440facc10cdcf9626ac5" member_type: VOTER last_known_addr { host: "127.12.138.194" port: 35029 } } peers { permanent_uuid: "44daa1d55934407699906055b6e717b7" member_type: VOTER last_known_addr { host: "127.12.138.195" port: 46619 } }
I20260811 02:56:04.264134 657 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4595d2f8cd984c96ae007da15d22bd49" candidate_uuid: "279f9395b2d54df2a22f4a7a0b74244b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "59c854d7d7c5440facc10cdcf9626ac5" is_pre_election: true
I20260811 02:56:04.264223 807 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4595d2f8cd984c96ae007da15d22bd49" candidate_uuid: "279f9395b2d54df2a22f4a7a0b74244b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "44daa1d55934407699906055b6e717b7" is_pre_election: true
I20260811 02:56:04.264418 657 raft_consensus.cc:2454] T 4595d2f8cd984c96ae007da15d22bd49 P 59c854d7d7c5440facc10cdcf9626ac5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 279f9395b2d54df2a22f4a7a0b74244b in term 0.
I20260811 02:56:04.264464 807 raft_consensus.cc:2454] T 4595d2f8cd984c96ae007da15d22bd49 P 44daa1d55934407699906055b6e717b7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 279f9395b2d54df2a22f4a7a0b74244b in term 0.
I20260811 02:56:04.264978 436 leader_election.cc:302] T 4595d2f8cd984c96ae007da15d22bd49 P 279f9395b2d54df2a22f4a7a0b74244b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 279f9395b2d54df2a22f4a7a0b74244b, 44daa1d55934407699906055b6e717b7, 59c854d7d7c5440facc10cdcf9626ac5; no voters:
I20260811 02:56:04.265275 890 raft_consensus.cc:2781] T 4595d2f8cd984c96ae007da15d22bd49 P 279f9395b2d54df2a22f4a7a0b74244b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 02:56:04.265398 890 raft_consensus.cc:482] T 4595d2f8cd984c96ae007da15d22bd49 P 279f9395b2d54df2a22f4a7a0b74244b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 02:56:04.265352 889 leader_election.cc:288] T 4595d2f8cd984c96ae007da15d22bd49 P 59c854d7d7c5440facc10cdcf9626ac5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 279f9395b2d54df2a22f4a7a0b74244b (127.12.138.193:39457), 44daa1d55934407699906055b6e717b7 (127.12.138.195:46619)
I20260811 02:56:04.265467 890 raft_consensus.cc:3037] T 4595d2f8cd984c96ae007da15d22bd49 P 279f9395b2d54df2a22f4a7a0b74244b [term 0 FOLLOWER]: Advancing to term 1
I20260811 02:56:04.266875 890 raft_consensus.cc:504] T 4595d2f8cd984c96ae007da15d22bd49 P 279f9395b2d54df2a22f4a7a0b74244b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "279f9395b2d54df2a22f4a7a0b74244b" member_type: VOTER last_known_addr { host: "127.12.138.193" port: 39457 } } peers { permanent_uuid: "59c854d7d7c5440facc10cdcf9626ac5" member_type: VOTER last_known_addr { host: "127.12.138.194" port: 35029 } } peers { permanent_uuid: "44daa1d55934407699906055b6e717b7" member_type: VOTER last_known_addr { host: "127.12.138.195" port: 46619 } }
I20260811 02:56:04.267231 890 leader_election.cc:288] T 4595d2f8cd984c96ae007da15d22bd49 P 279f9395b2d54df2a22f4a7a0b74244b [CANDIDATE]: Term 1 election: Requested vote from peers 59c854d7d7c5440facc10cdcf9626ac5 (127.12.138.194:35029), 44daa1d55934407699906055b6e717b7 (127.12.138.195:46619)
I20260811 02:56:04.267592 657 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4595d2f8cd984c96ae007da15d22bd49" candidate_uuid: "279f9395b2d54df2a22f4a7a0b74244b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "59c854d7d7c5440facc10cdcf9626ac5"
I20260811 02:56:04.267629 807 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4595d2f8cd984c96ae007da15d22bd49" candidate_uuid: "279f9395b2d54df2a22f4a7a0b74244b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "44daa1d55934407699906055b6e717b7"
I20260811 02:56:04.267730 657 raft_consensus.cc:3037] T 4595d2f8cd984c96ae007da15d22bd49 P 59c854d7d7c5440facc10cdcf9626ac5 [term 0 FOLLOWER]: Advancing to term 1
I20260811 02:56:04.267735 807 raft_consensus.cc:3037] T 4595d2f8cd984c96ae007da15d22bd49 P 44daa1d55934407699906055b6e717b7 [term 0 FOLLOWER]: Advancing to term 1
I20260811 02:56:04.269058 807 raft_consensus.cc:2454] T 4595d2f8cd984c96ae007da15d22bd49 P 44daa1d55934407699906055b6e717b7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 279f9395b2d54df2a22f4a7a0b74244b in term 1.
I20260811 02:56:04.269063 657 raft_consensus.cc:2454] T 4595d2f8cd984c96ae007da15d22bd49 P 59c854d7d7c5440facc10cdcf9626ac5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 279f9395b2d54df2a22f4a7a0b74244b in term 1.
I20260811 02:56:04.269469 436 leader_election.cc:302] T 4595d2f8cd984c96ae007da15d22bd49 P 279f9395b2d54df2a22f4a7a0b74244b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 279f9395b2d54df2a22f4a7a0b74244b, 44daa1d55934407699906055b6e717b7, 59c854d7d7c5440facc10cdcf9626ac5; no voters:
I20260811 02:56:04.269624 890 raft_consensus.cc:2781] T 4595d2f8cd984c96ae007da15d22bd49 P 279f9395b2d54df2a22f4a7a0b74244b [term 1 FOLLOWER]: Leader election won for term 1
I20260811 02:56:04.269876 890 raft_consensus.cc:686] T 4595d2f8cd984c96ae007da15d22bd49 P 279f9395b2d54df2a22f4a7a0b74244b [term 1 LEADER]: Becoming Leader. State: Replica: 279f9395b2d54df2a22f4a7a0b74244b, State: Running, Role: LEADER
I20260811 02:56:04.270237 890 consensus_queue.cc:237] T 4595d2f8cd984c96ae007da15d22bd49 P 279f9395b2d54df2a22f4a7a0b74244b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "279f9395b2d54df2a22f4a7a0b74244b" member_type: VOTER last_known_addr { host: "127.12.138.193" port: 39457 } } peers { permanent_uuid: "59c854d7d7c5440facc10cdcf9626ac5" member_type: VOTER last_known_addr { host: "127.12.138.194" port: 35029 } } peers { permanent_uuid: "44daa1d55934407699906055b6e717b7" member_type: VOTER last_known_addr { host: "127.12.138.195" port: 46619 } }
I20260811 02:56:04.270462 879 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:56:04.265689 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.138.194:42689 (local address 127.12.138.195:46619)
0811 02:56:04.265818 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0811 02:56:04.265822 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 02:56:04.266059 (+ 237us) server_negotiation.cc:407] Connection header received
0811 02:56:04.266181 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:56:04.266186 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:56:04.266234 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:56:04.266302 (+ 68us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 02:56:04.266663 (+ 361us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:56:04.267340 (+ 677us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:56:04.269200 (+ 1860us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:56:04.269786 (+ 586us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:56:04.269908 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:56:04.269968 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0811 02:56:04.270276 (+ 308us) server_negotiation.cc:299] Negotiation successful
0811 02:56:04.270347 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260811 02:56:04.270823 894 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:56:04.265641 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.138.195:46619 (local address 127.12.138.194:42689)
0811 02:56:04.265944 (+ 303us) negotiation.cc:107] Waiting for socket to connect
0811 02:56:04.265960 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 02:56:04.266078 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:56:04.266340 (+ 262us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:56:04.266344 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:56:04.266367 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 02:56:04.266519 (+ 152us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:56:04.266524 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:56:04.267471 (+ 947us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:56:04.267474 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:56:04.269024 (+ 1550us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:56:04.269036 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:56:04.270031 (+ 995us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:56:04.270035 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:56:04.270133 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:56:04.270157 (+ 24us) client_negotiation.cc:769] Sending connection context
0811 02:56:04.270581 (+ 424us) client_negotiation.cc:240] Negotiation successful
0811 02:56:04.270667 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":240,"mutex_wait_us":1,"thread_start_us":91,"threads_started":1}
I20260811 02:56:04.271281 875 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:56:04.267068 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.138.194:33483 (local address 127.12.138.193:39457)
0811 02:56:04.267176 (+ 108us) server_negotiation.cc:206] Beginning negotiation
0811 02:56:04.267179 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 02:56:04.267217 (+ 38us) server_negotiation.cc:407] Connection header received
0811 02:56:04.267259 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:56:04.267262 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:56:04.267312 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:56:04.267382 (+ 70us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 02:56:04.267881 (+ 499us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:56:04.268567 (+ 686us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:56:04.270045 (+ 1478us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:56:04.270655 (+ 610us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:56:04.270743 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:56:04.270785 (+ 42us) server_negotiation.cc:1035] Waiting for connection context
0811 02:56:04.271109 (+ 324us) server_negotiation.cc:299] Negotiation successful
0811 02:56:04.271184 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260811 02:56:04.271298 893 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:56:04.265696 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.138.193:39457 (local address 127.12.138.194:33483)
0811 02:56:04.266044 (+ 348us) negotiation.cc:107] Waiting for socket to connect
0811 02:56:04.266056 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 02:56:04.266122 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:56:04.267596 (+ 1474us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:56:04.267599 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:56:04.267619 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 02:56:04.267745 (+ 126us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:56:04.267751 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:56:04.268705 (+ 954us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:56:04.268708 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:56:04.269910 (+ 1202us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:56:04.269920 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:56:04.270881 (+ 961us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:56:04.270885 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:56:04.270995 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:56:04.271018 (+ 23us) client_negotiation.cc:769] Sending connection context
0811 02:56:04.271093 (+ 75us) client_negotiation.cc:240] Negotiation successful
0811 02:56:04.271167 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":289,"thread_start_us":90,"threads_started":1}
I20260811 02:56:04.271780 508 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4595d2f8cd984c96ae007da15d22bd49" candidate_uuid: "59c854d7d7c5440facc10cdcf9626ac5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "279f9395b2d54df2a22f4a7a0b74244b" is_pre_election: true
I20260811 02:56:04.271896 807 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4595d2f8cd984c96ae007da15d22bd49" candidate_uuid: "59c854d7d7c5440facc10cdcf9626ac5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "44daa1d55934407699906055b6e717b7" is_pre_election: true
I20260811 02:56:04.272125 807 raft_consensus.cc:2379] T 4595d2f8cd984c96ae007da15d22bd49 P 44daa1d55934407699906055b6e717b7 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 59c854d7d7c5440facc10cdcf9626ac5 in current term 1: Already voted for candidate 279f9395b2d54df2a22f4a7a0b74244b in this term.
I20260811 02:56:04.272593 584 leader_election.cc:302] T 4595d2f8cd984c96ae007da15d22bd49 P 59c854d7d7c5440facc10cdcf9626ac5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 59c854d7d7c5440facc10cdcf9626ac5; no voters: 279f9395b2d54df2a22f4a7a0b74244b, 44daa1d55934407699906055b6e717b7
I20260811 02:56:04.272873 889 raft_consensus.cc:2726] T 4595d2f8cd984c96ae007da15d22bd49 P 59c854d7d7c5440facc10cdcf9626ac5 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260811 02:56:04.274608 343 catalog_manager.cc:5495] T 4595d2f8cd984c96ae007da15d22bd49 P 279f9395b2d54df2a22f4a7a0b74244b reported cstate change: term changed from 0 to 1, leader changed from <none> to 279f9395b2d54df2a22f4a7a0b74244b (127.12.138.193). New cstate: current_term: 1 leader_uuid: "279f9395b2d54df2a22f4a7a0b74244b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "279f9395b2d54df2a22f4a7a0b74244b" member_type: VOTER last_known_addr { host: "127.12.138.193" port: 39457 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "59c854d7d7c5440facc10cdcf9626ac5" member_type: VOTER last_known_addr { host: "127.12.138.194" port: 35029 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "44daa1d55934407699906055b6e717b7" member_type: VOTER last_known_addr { host: "127.12.138.195" port: 46619 } health_report { overall_health: UNKNOWN } } }
I20260811 02:56:04.345060 875 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:56:04.341196 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38504 (local address 127.12.138.193:39457)
0811 02:56:04.341343 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0811 02:56:04.341352 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 02:56:04.341366 (+ 14us) server_negotiation.cc:407] Connection header received
0811 02:56:04.341483 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:56:04.341485 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:56:04.341541 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:56:04.341658 (+ 117us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 02:56:04.341954 (+ 296us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:56:04.342463 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:56:04.343418 (+ 955us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:56:04.343625 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:56:04.343704 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:56:04.344087 (+ 383us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 02:56:04.344515 (+ 428us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 02:56:04.344609 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0811 02:56:04.344882 (+ 273us) server_negotiation.cc:299] Negotiation successful
0811 02:56:04.344943 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260811 02:56:04.351541 807 raft_consensus.cc:1261] T 4595d2f8cd984c96ae007da15d22bd49 P 44daa1d55934407699906055b6e717b7 [term 1 FOLLOWER]: Refusing update from remote peer 279f9395b2d54df2a22f4a7a0b74244b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 02:56:04.351691 657 raft_consensus.cc:1261] T 4595d2f8cd984c96ae007da15d22bd49 P 59c854d7d7c5440facc10cdcf9626ac5 [term 1 FOLLOWER]: Refusing update from remote peer 279f9395b2d54df2a22f4a7a0b74244b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 02:56:04.352290 890 consensus_queue.cc:1035] T 4595d2f8cd984c96ae007da15d22bd49 P 279f9395b2d54df2a22f4a7a0b74244b [LEADER]: Connected to new peer: Peer: permanent_uuid: "44daa1d55934407699906055b6e717b7" member_type: VOTER last_known_addr { host: "127.12.138.195" port: 46619 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 02:56:04.352502 895 consensus_queue.cc:1035] T 4595d2f8cd984c96ae007da15d22bd49 P 279f9395b2d54df2a22f4a7a0b74244b [LEADER]: Connected to new peer: Peer: permanent_uuid: "59c854d7d7c5440facc10cdcf9626ac5" member_type: VOTER last_known_addr { host: "127.12.138.194" port: 35029 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 02:56:04.360307 898 mvcc.cc:205] Tried to move back new op lower bound from 7317163885976252416 to 7317163885654061056. Current Snapshot: MvccSnapshot[applied={T|T < 7317163885976252416}]
I20260811 02:56:04.363826 899 mvcc.cc:205] Tried to move back new op lower bound from 7317163885976252416 to 7317163885654061056. Current Snapshot: MvccSnapshot[applied={T|T < 7317163885976252416 or (T in {7317163885976252416})}]
I20260811 02:56:04.368970 344 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44750:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 02:56:04.369179 344 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44750:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 02:56:04.371694 344 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 4ec9fb2d07bf4bb38266323d6d36b408: Sending DeleteTablet for 3 replicas of tablet 4595d2f8cd984c96ae007da15d22bd49
I20260811 02:56:04.372563 636 tablet_service.cc:1514] Processing DeleteTablet for tablet 4595d2f8cd984c96ae007da15d22bd49 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 02:56:04 UTC) from {username='kudu', principal='kudu/127.12.138.254@KRBTEST.COM'} at 127.0.0.1:53802
I20260811 02:56:04.372565 485 tablet_service.cc:1514] Processing DeleteTablet for tablet 4595d2f8cd984c96ae007da15d22bd49 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 02:56:04 UTC) from {username='kudu', principal='kudu/127.12.138.254@KRBTEST.COM'} at 127.0.0.1:38500
I20260811 02:56:04.373133 783 tablet_service.cc:1514] Processing DeleteTablet for tablet 4595d2f8cd984c96ae007da15d22bd49 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 02:56:04 UTC) from {username='kudu', principal='kudu/127.12.138.254@KRBTEST.COM'} at 127.0.0.1:58296
I20260811 02:56:04.373320 909 tablet_replica.cc:330] T 4595d2f8cd984c96ae007da15d22bd49 P 279f9395b2d54df2a22f4a7a0b74244b: stopping tablet replica
I20260811 02:56:04.373631 909 raft_consensus.cc:2229] T 4595d2f8cd984c96ae007da15d22bd49 P 279f9395b2d54df2a22f4a7a0b74244b [term 1 LEADER]: Raft consensus shutting down.
I20260811 02:56:04.373662 911 tablet_replica.cc:330] T 4595d2f8cd984c96ae007da15d22bd49 P 44daa1d55934407699906055b6e717b7: stopping tablet replica
I20260811 02:56:04.373957 911 raft_consensus.cc:2229] T 4595d2f8cd984c96ae007da15d22bd49 P 44daa1d55934407699906055b6e717b7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 02:56:04.374047 909 raft_consensus.cc:2258] T 4595d2f8cd984c96ae007da15d22bd49 P 279f9395b2d54df2a22f4a7a0b74244b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 02:56:04.374286 908 tablet_replica.cc:330] T 4595d2f8cd984c96ae007da15d22bd49 P 59c854d7d7c5440facc10cdcf9626ac5: stopping tablet replica
I20260811 02:56:04.374294 911 raft_consensus.cc:2258] T 4595d2f8cd984c96ae007da15d22bd49 P 44daa1d55934407699906055b6e717b7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 02:56:04.374444 12843 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDawEwS/build/debug/bin/kudu with pid 416
I20260811 02:56:04.374607 908 raft_consensus.cc:2229] T 4595d2f8cd984c96ae007da15d22bd49 P 59c854d7d7c5440facc10cdcf9626ac5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 02:56:04.374909 908 raft_consensus.cc:2258] T 4595d2f8cd984c96ae007da15d22bd49 P 59c854d7d7c5440facc10cdcf9626ac5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 02:56:04.376722 911 ts_tablet_manager.cc:1905] T 4595d2f8cd984c96ae007da15d22bd49 P 44daa1d55934407699906055b6e717b7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 02:56:04.380491 911 ts_tablet_manager.cc:1918] T 4595d2f8cd984c96ae007da15d22bd49 P 44daa1d55934407699906055b6e717b7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 02:56:04.380599 911 log.cc:1196] T 4595d2f8cd984c96ae007da15d22bd49 P 44daa1d55934407699906055b6e717b7: Deleting WAL directory at /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786416707705494-12843-0/minicluster-data/ts-2/wal/wals/4595d2f8cd984c96ae007da15d22bd49
I20260811 02:56:04.380924 911 ts_tablet_manager.cc:1939] T 4595d2f8cd984c96ae007da15d22bd49 P 44daa1d55934407699906055b6e717b7: Deleting consensus metadata
I20260811 02:56:04.381060 908 ts_tablet_manager.cc:1905] T 4595d2f8cd984c96ae007da15d22bd49 P 59c854d7d7c5440facc10cdcf9626ac5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 02:56:04.382395 329 catalog_manager.cc:4841] TS 44daa1d55934407699906055b6e717b7 (127.12.138.195:46619): tablet 4595d2f8cd984c96ae007da15d22bd49 (table test-table [id=6a9790ff9aab4d52887c2cb5ddcc019c]) successfully deleted
I20260811 02:56:04.383343 908 ts_tablet_manager.cc:1918] T 4595d2f8cd984c96ae007da15d22bd49 P 59c854d7d7c5440facc10cdcf9626ac5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 02:56:04.383425 908 log.cc:1196] T 4595d2f8cd984c96ae007da15d22bd49 P 59c854d7d7c5440facc10cdcf9626ac5: Deleting WAL directory at /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786416707705494-12843-0/minicluster-data/ts-1/wal/wals/4595d2f8cd984c96ae007da15d22bd49
I20260811 02:56:04.384003 908 ts_tablet_manager.cc:1939] T 4595d2f8cd984c96ae007da15d22bd49 P 59c854d7d7c5440facc10cdcf9626ac5: Deleting consensus metadata
I20260811 02:56:04.385074 328 catalog_manager.cc:4841] TS 59c854d7d7c5440facc10cdcf9626ac5 (127.12.138.194:35029): tablet 4595d2f8cd984c96ae007da15d22bd49 (table test-table [id=6a9790ff9aab4d52887c2cb5ddcc019c]) successfully deleted
W20260811 02:56:04.386750 327 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.12.138.193:39457 (error 108)
I20260811 02:56:04.387048 12843 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDawEwS/build/debug/bin/kudu with pid 565
I20260811 02:56:04.388319 877 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:56:04.387876 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.138.193:39457 (local address 127.0.0.1:38514)
0811 02:56:04.388111 (+ 235us) negotiation.cc:107] Waiting for socket to connect
0811 02:56:04.388205 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.138.193:39457: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":129}
W20260811 02:56:04.388506 327 catalog_manager.cc:4584] TS 279f9395b2d54df2a22f4a7a0b74244b (127.12.138.193:39457): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 4595d2f8cd984c96ae007da15d22bd49: Network error: Client connection negotiation failed: client connection to 127.12.138.193:39457: connect: Connection refused (error 111)
I20260811 02:56:04.396628 12843 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDawEwS/build/debug/bin/kudu with pid 712
I20260811 02:56:04.404098 12843 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDawEwS/build/debug/bin/kudu with pid 307
2026-08-11T02:56:04Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/0 (2801 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/1
2026-08-11T02:56:04Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T02:56:04Z Disabled control of system clock
I20260811 02:56:04.439545 12843 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDawEwS/build/debug/bin/kudu
/tmp/dist-test-taskDawEwS/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786416707705494-12843-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786416707705494-12843-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786416707705494-12843-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786416707705494-12843-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.138.254:41141
--webserver_interface=127.12.138.254
--webserver_port=0
--builtin_ntp_servers=127.12.138.212:35353
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.12.138.254:41141
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260811 02:56:04.561672 917 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 02:56:04.561962 917 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 02:56:04.562000 917 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 02:56:04.566121 917 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 02:56:04.566188 917 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 02:56:04.566205 917 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 02:56:04.566221 917 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 02:56:04.566237 917 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 02:56:04.571256 917 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.138.212:35353
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786416707705494-12843-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786416707705494-12843-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.12.138.254:41141
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.138.254:41141
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786416707705494-12843-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.12.138.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.917
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786416707705494-12843-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 02:09:49 UTC on ccycloud.quasar-fsnqqc.root.comops.site
build id 4579570
I20260811 02:56:04.572460 917 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 02:56:04.574000 917 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 02:56:04.583627 926 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:56:04.583662 924 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:56:04.583827 917 server_base.cc:1034] running on GCE node
W20260811 02:56:04.583678 923 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:56:04.584554 917 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 02:56:04.585850 917 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 02:56:04.587016 917 hybrid_clock.cc:648] HybridClock initialized: now 1786416964586990 us; error 39 us; skew 500 ppm
I20260811 02:56:04.596853 917 webserver.cc:460] Webserver started at http://127.12.138.254:34343/ using document root <none> and password file <none>
I20260811 02:56:04.597568 917 fs_manager.cc:359] Metadata directory not provided
I20260811 02:56:04.597668 917 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 02:56:04.597922 917 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 02:56:04.600143 917 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786416707705494-12843-0/minicluster-data/master-0/data/instance:
uuid: "f8ba95b215cc402aa2b89b6bf227168d"
format_stamp: "Formatted at 2026-08-11 02:56:04 on dist-test-slave-9gcw"
server_key: "e4cc1af1b626ea79051a5a4c357c22eb"
server_key_iv: "19684e74e63522e4be17d43eb5e5e73f"
server_key_version: "encryptionkey@0"
I20260811 02:56:04.600755 917 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786416707705494-12843-0/minicluster-data/master-0/wal/instance:
uuid: "f8ba95b215cc402aa2b89b6bf227168d"
format_stamp: "Formatted at 2026-08-11 02:56:04 on dist-test-slave-9gcw"
server_key: "e4cc1af1b626ea79051a5a4c357c22eb"
server_key_iv: "19684e74e63522e4be17d43eb5e5e73f"
server_key_version: "encryptionkey@0"
I20260811 02:56:04.604866 917 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260811 02:56:04.607736 933 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 02:56:04.609243 917 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 02:56:04.609387 917 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786416707705494-12843-0/minicluster-data/master-0/data,/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786416707705494-12843-0/minicluster-data/master-0/wal
uuid: "f8ba95b215cc402aa2b89b6bf227168d"
format_stamp: "Formatted at 2026-08-11 02:56:04 on dist-test-slave-9gcw"
server_key: "e4cc1af1b626ea79051a5a4c357c22eb"
server_key_iv: "19684e74e63522e4be17d43eb5e5e73f"
server_key_version: "encryptionkey@0"
I20260811 02:56:04.609516 917 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786416707705494-12843-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786416707705494-12843-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786416707705494-12843-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 02:56:04.651459 917 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 02:56:04.652268 917 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 02:56:04.652495 917 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 02:56:04.659920 917 rpc_server.cc:307] RPC server started. Bound to: 127.12.138.254:41141
I20260811 02:56:04.659931 990 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.138.254:41141 every 8 connection(s)
I20260811 02:56:04.660971 917 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786416707705494-12843-0/minicluster-data/master-0/data/info.pb
I20260811 02:56:04.664310 991 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 02:56:04.667112 12843 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDawEwS/build/debug/bin/kudu as pid 917
I20260811 02:56:04.667268 12843 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786416707705494-12843-0/minicluster-data/master-0/wal/instance
I20260811 02:56:04.667614 12843 external_mini_cluster.cc:1428] Setting key cee630db9c0cc0532f3070661f5608c1
I20260811 02:56:04.671051 991 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f8ba95b215cc402aa2b89b6bf227168d: Bootstrap starting.
I20260811 02:56:04.673874 991 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f8ba95b215cc402aa2b89b6bf227168d: Neither blocks nor log segments found. Creating new log.
I20260811 02:56:04.674675 991 log.cc:824] T 00000000000000000000000000000000 P f8ba95b215cc402aa2b89b6bf227168d: Log is configured to *not* fsync() on all Append() calls
I20260811 02:56:04.677186 991 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f8ba95b215cc402aa2b89b6bf227168d: No bootstrap required, opened a new log
I20260811 02:56:04.678921 994 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:56:04.669242 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36796 (local address 127.12.138.254:41141)
0811 02:56:04.670014 (+ 772us) server_negotiation.cc:206] Beginning negotiation
0811 02:56:04.670035 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0811 02:56:04.670081 (+ 46us) server_negotiation.cc:407] Connection header received
0811 02:56:04.670910 (+ 829us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:56:04.670934 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:56:04.671464 (+ 530us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:56:04.671834 (+ 370us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:56:04.672365 (+ 531us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:56:04.673869 (+ 1504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:56:04.675028 (+ 1159us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:56:04.675635 (+ 607us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:56:04.675805 (+ 170us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:56:04.676759 (+ 954us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:56:04.676765 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:56:04.676780 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 02:56:04.677338 (+ 558us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:56:04.677413 (+ 75us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 02:56:04.677440 (+ 27us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:56:04.677544 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0811 02:56:04.677724 (+ 180us) server_negotiation.cc:299] Negotiation successful
0811 02:56:04.677995 (+ 271us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":311,"thread_start_us":132,"threads_started":1}
I20260811 02:56:04.681536 991 raft_consensus.cc:348] T 00000000000000000000000000000000 P f8ba95b215cc402aa2b89b6bf227168d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f8ba95b215cc402aa2b89b6bf227168d" member_type: VOTER last_known_addr { host: "127.12.138.254" port: 41141 } }
I20260811 02:56:04.681762 991 raft_consensus.cc:374] T 00000000000000000000000000000000 P f8ba95b215cc402aa2b89b6bf227168d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 02:56:04.681799 991 raft_consensus.cc:729] T 00000000000000000000000000000000 P f8ba95b215cc402aa2b89b6bf227168d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f8ba95b215cc402aa2b89b6bf227168d, State: Initialized, Role: FOLLOWER
I20260811 02:56:04.682235 991 consensus_queue.cc:260] T 00000000000000000000000000000000 P f8ba95b215cc402aa2b89b6bf227168d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f8ba95b215cc402aa2b89b6bf227168d" member_type: VOTER last_known_addr { host: "127.12.138.254" port: 41141 } }
I20260811 02:56:04.682360 991 raft_consensus.cc:388] T 00000000000000000000000000000000 P f8ba95b215cc402aa2b89b6bf227168d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 02:56:04.682410 991 raft_consensus.cc:482] T 00000000000000000000000000000000 P f8ba95b215cc402aa2b89b6bf227168d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 02:56:04.682483 991 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f8ba95b215cc402aa2b89b6bf227168d [term 0 FOLLOWER]: Advancing to term 1
I20260811 02:56:04.683632 991 raft_consensus.cc:504] T 00000000000000000000000000000000 P f8ba95b215cc402aa2b89b6bf227168d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f8ba95b215cc402aa2b89b6bf227168d" member_type: VOTER last_known_addr { host: "127.12.138.254" port: 41141 } }
I20260811 02:56:04.684038 991 leader_election.cc:302] T 00000000000000000000000000000000 P f8ba95b215cc402aa2b89b6bf227168d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f8ba95b215cc402aa2b89b6bf227168d; no voters:
I20260811 02:56:04.684367 991 leader_election.cc:288] T 00000000000000000000000000000000 P f8ba95b215cc402aa2b89b6bf227168d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 02:56:04.684589 996 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f8ba95b215cc402aa2b89b6bf227168d [term 1 FOLLOWER]: Leader election won for term 1
I20260811 02:56:04.684967 996 raft_consensus.cc:686] T 00000000000000000000000000000000 P f8ba95b215cc402aa2b89b6bf227168d [term 1 LEADER]: Becoming Leader. State: Replica: f8ba95b215cc402aa2b89b6bf227168d, State: Running, Role: LEADER
I20260811 02:56:04.685348 996 consensus_queue.cc:237] T 00000000000000000000000000000000 P f8ba95b215cc402aa2b89b6bf227168d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f8ba95b215cc402aa2b89b6bf227168d" member_type: VOTER last_known_addr { host: "127.12.138.254" port: 41141 } }
I20260811 02:56:04.685719 991 sys_catalog.cc:564] T 00000000000000000000000000000000 P f8ba95b215cc402aa2b89b6bf227168d [sys.catalog]: configured and running, proceeding with master startup.
I20260811 02:56:04.687127 997 sys_catalog.cc:455] T 00000000000000000000000000000000 P f8ba95b215cc402aa2b89b6bf227168d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f8ba95b215cc402aa2b89b6bf227168d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f8ba95b215cc402aa2b89b6bf227168d" member_type: VOTER last_known_addr { host: "127.12.138.254" port: 41141 } } }
I20260811 02:56:04.687278 997 sys_catalog.cc:458] T 00000000000000000000000000000000 P f8ba95b215cc402aa2b89b6bf227168d [sys.catalog]: This master's current role is: LEADER
I20260811 02:56:04.687698 1011 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 02:56:04.688102 999 sys_catalog.cc:455] T 00000000000000000000000000000000 P f8ba95b215cc402aa2b89b6bf227168d [sys.catalog]: SysCatalogTable state changed. Reason: New leader f8ba95b215cc402aa2b89b6bf227168d. Latest consensus state: current_term: 1 leader_uuid: "f8ba95b215cc402aa2b89b6bf227168d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f8ba95b215cc402aa2b89b6bf227168d" member_type: VOTER last_known_addr { host: "127.12.138.254" port: 41141 } } }
I20260811 02:56:04.688549 999 sys_catalog.cc:458] T 00000000000000000000000000000000 P f8ba95b215cc402aa2b89b6bf227168d [sys.catalog]: This master's current role is: LEADER
I20260811 02:56:04.691004 1011 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 02:56:04.696604 1011 catalog_manager.cc:1347] Generated new cluster ID: ed1bbe2abb5b4ad5a2411342a75d2c7c
I20260811 02:56:04.696704 1011 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 02:56:04.710734 1011 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 02:56:04.711647 1011 catalog_manager.cc:1504] Loading token signing keys...
I20260811 02:56:04.720504 1011 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f8ba95b215cc402aa2b89b6bf227168d: Generated new TSK 0
I20260811 02:56:04.721444 1011 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 02:56:04.733718 12843 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDawEwS/build/debug/bin/kudu
/tmp/dist-test-taskDawEwS/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786416707705494-12843-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786416707705494-12843-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786416707705494-12843-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786416707705494-12843-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.138.193:0
--local_ip_for_outbound_sockets=127.12.138.193
--webserver_interface=127.12.138.193
--webserver_port=0
--tserver_master_addrs=127.12.138.254:41141
--builtin_ntp_servers=127.12.138.212:35353
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260811 02:56:04.863761 1019 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 02:56:04.864054 1019 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 02:56:04.864130 1019 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 02:56:04.868193 1019 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 02:56:04.868260 1019 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 02:56:04.868317 1019 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.138.193
I20260811 02:56:04.873032 1019 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.138.212:35353
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786416707705494-12843-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786416707705494-12843-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.138.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786416707705494-12843-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.12.138.193
--webserver_port=0
--tserver_master_addrs=127.12.138.254:41141
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1019
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.138.193
--log_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786416707705494-12843-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 02:09:49 UTC on ccycloud.quasar-fsnqqc.root.comops.site
build id 4579570
I20260811 02:56:04.874217 1019 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 02:56:04.875641 1019 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 02:56:04.886332 1027 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:56:04.886343 1030 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:56:04.886349 1026 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:56:04.886943 1019 server_base.cc:1034] running on GCE node
I20260811 02:56:04.887559 1019 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 02:56:04.888365 1019 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 02:56:04.889599 1019 hybrid_clock.cc:648] HybridClock initialized: now 1786416964889581 us; error 40 us; skew 500 ppm
I20260811 02:56:04.891883 1019 webserver.cc:460] Webserver started at http://127.12.138.193:39981/ using document root <none> and password file <none>
I20260811 02:56:04.892652 1019 fs_manager.cc:359] Metadata directory not provided
I20260811 02:56:04.892773 1019 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 02:56:04.893071 1019 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 02:56:04.896417 1019 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786416707705494-12843-0/minicluster-data/ts-0/data/instance:
uuid: "f3862759323e4abe84a20f26e81a3c8a"
format_stamp: "Formatted at 2026-08-11 02:56:04 on dist-test-slave-9gcw"
server_key: "78a496dca1fd7de2fa1e183fe55825c7"
server_key_iv: "481884366884fba0d15c31904ab8f795"
server_key_version: "encryptionkey@0"
I20260811 02:56:04.897195 1019 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786416707705494-12843-0/minicluster-data/ts-0/wal/instance:
uuid: "f3862759323e4abe84a20f26e81a3c8a"
format_stamp: "Formatted at 2026-08-11 02:56:04 on dist-test-slave-9gcw"
server_key: "78a496dca1fd7de2fa1e183fe55825c7"
server_key_iv: "481884366884fba0d15c31904ab8f795"
server_key_version: "encryptionkey@0"
I20260811 02:56:04.901621 1019 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260811 02:56:04.904584 1037 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 02:56:04.906150 1019 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 02:56:04.906312 1019 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786416707705494-12843-0/minicluster-data/ts-0/data,/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786416707705494-12843-0/minicluster-data/ts-0/wal
uuid: "f3862759323e4abe84a20f26e81a3c8a"
format_stamp: "Formatted at 2026-08-11 02:56:04 on dist-test-slave-9gcw"
server_key: "78a496dca1fd7de2fa1e183fe55825c7"
server_key_iv: "481884366884fba0d15c31904ab8f795"
server_key_version: "encryptionkey@0"
I20260811 02:56:04.906450 1019 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786416707705494-12843-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786416707705494-12843-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786416707705494-12843-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 02:56:04.937772 1019 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 02:56:04.938587 1019 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 02:56:04.938832 1019 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 02:56:04.939502 1019 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 02:56:04.940567 1019 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 02:56:04.940647 1019 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 02:56:04.940727 1019 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 02:56:04.940774 1019 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 02:56:04.951376 1019 rpc_server.cc:307] RPC server started. Bound to: 127.12.138.193:39445
I20260811 02:56:04.951465 1163 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.138.193:39445 every 8 connection(s)
I20260811 02:56:04.952462 1019 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786416707705494-12843-0/minicluster-data/ts-0/data/info.pb
I20260811 02:56:04.952770 12843 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDawEwS/build/debug/bin/kudu as pid 1019
I20260811 02:56:04.952935 12843 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786416707705494-12843-0/minicluster-data/ts-0/wal/instance
I20260811 02:56:04.953339 12843 external_mini_cluster.cc:1428] Setting key 528ebcf68bd757c8d0343215cf720fed
I20260811 02:56:04.955703 12843 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDawEwS/build/debug/bin/kudu
/tmp/dist-test-taskDawEwS/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786416707705494-12843-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786416707705494-12843-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786416707705494-12843-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786416707705494-12843-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.138.194:0
--local_ip_for_outbound_sockets=127.12.138.194
--webserver_interface=127.12.138.194
--webserver_port=0
--tserver_master_addrs=127.12.138.254:41141
--builtin_ntp_servers=127.12.138.212:35353
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260811 02:56:04.964079 994 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:56:04.954641 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.138.193:39859 (local address 127.12.138.254:41141)
0811 02:56:04.954781 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0811 02:56:04.954785 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 02:56:04.956320 (+ 1535us) server_negotiation.cc:407] Connection header received
0811 02:56:04.958326 (+ 2006us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:56:04.958328 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:56:04.958380 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:56:04.958456 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:56:04.959772 (+ 1316us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:56:04.960382 (+ 610us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:56:04.961787 (+ 1405us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:56:04.962048 (+ 261us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:56:04.962181 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:56:04.963511 (+ 1330us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:56:04.963515 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:56:04.963521 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 02:56:04.963541 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:56:04.963562 (+ 21us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 02:56:04.963571 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:56:04.963657 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0811 02:56:04.963847 (+ 190us) server_negotiation.cc:299] Negotiation successful
0811 02:56:04.963918 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260811 02:56:04.965014 1166 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:56:04.955273 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.138.254:41141 (local address 127.12.138.193:39859)
0811 02:56:04.956157 (+ 884us) negotiation.cc:107] Waiting for socket to connect
0811 02:56:04.956217 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0811 02:56:04.958044 (+ 1827us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:56:04.958670 (+ 626us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:56:04.958681 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:56:04.958809 (+ 128us) client_negotiation.cc:189] Negotiated authn=SASL
0811 02:56:04.959608 (+ 799us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:56:04.959627 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:56:04.960532 (+ 905us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:56:04.960536 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:56:04.961645 (+ 1109us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:56:04.961656 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:56:04.962226 (+ 570us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:56:04.962231 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:56:04.962442 (+ 211us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:56:04.963224 (+ 782us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 02:56:04.963226 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0811 02:56:04.963269 (+ 43us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 02:56:04.963340 (+ 71us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 02:56:04.963673 (+ 333us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 02:56:04.963686 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 02:56:04.963698 (+ 12us) client_negotiation.cc:769] Sending connection context
0811 02:56:04.963835 (+ 137us) client_negotiation.cc:240] Negotiation successful
0811 02:56:04.964154 (+ 319us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":363,"thread_start_us":163,"threads_started":1}
I20260811 02:56:04.966553 1164 heartbeater.cc:347] Connected to a master server at 127.12.138.254:41141
I20260811 02:56:04.966944 1164 heartbeater.cc:464] Registering TS with master...
I20260811 02:56:04.967585 1164 heartbeater.cc:511] Master 127.12.138.254:41141 requested a full tablet report, sending...
I20260811 02:56:04.969503 953 ts_manager.cc:194] Registered new tserver with Master: f3862759323e4abe84a20f26e81a3c8a (127.12.138.193:39445)
I20260811 02:56:04.970785 953 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.12.138.193:39859
W20260811 02:56:05.078156 1167 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 02:56:05.078517 1167 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 02:56:05.078606 1167 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 02:56:05.083056 1167 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 02:56:05.083161 1167 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 02:56:05.083258 1167 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.138.194
I20260811 02:56:05.088600 1167 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.138.212:35353
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786416707705494-12843-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786416707705494-12843-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.138.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786416707705494-12843-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.12.138.194
--webserver_port=0
--tserver_master_addrs=127.12.138.254:41141
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1167
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.138.194
--log_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786416707705494-12843-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 02:09:49 UTC on ccycloud.quasar-fsnqqc.root.comops.site
build id 4579570
I20260811 02:56:05.089900 1167 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 02:56:05.091408 1167 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 02:56:05.102734 1175 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:56:05.102691 1177 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:56:05.102665 1173 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:56:05.103016 1167 server_base.cc:1034] running on GCE node
I20260811 02:56:05.103610 1167 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 02:56:05.104333 1167 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 02:56:05.105585 1167 hybrid_clock.cc:648] HybridClock initialized: now 1786416965105529 us; error 85 us; skew 500 ppm
I20260811 02:56:05.107906 1167 webserver.cc:460] Webserver started at http://127.12.138.194:38855/ using document root <none> and password file <none>
I20260811 02:56:05.108570 1167 fs_manager.cc:359] Metadata directory not provided
I20260811 02:56:05.108680 1167 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 02:56:05.108937 1167 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 02:56:05.111333 1167 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786416707705494-12843-0/minicluster-data/ts-1/data/instance:
uuid: "63c7ec3943ee4c3987785755411738fa"
format_stamp: "Formatted at 2026-08-11 02:56:05 on dist-test-slave-9gcw"
server_key: "dda92a3d92854c2e7ff86e891533128a"
server_key_iv: "a2470fe6845fbbd4e72941b40f215dc7"
server_key_version: "encryptionkey@0"
I20260811 02:56:05.111944 1167 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786416707705494-12843-0/minicluster-data/ts-1/wal/instance:
uuid: "63c7ec3943ee4c3987785755411738fa"
format_stamp: "Formatted at 2026-08-11 02:56:05 on dist-test-slave-9gcw"
server_key: "dda92a3d92854c2e7ff86e891533128a"
server_key_iv: "a2470fe6845fbbd4e72941b40f215dc7"
server_key_version: "encryptionkey@0"
I20260811 02:56:05.116151 1167 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.004s
I20260811 02:56:05.119035 1183 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 02:56:05.120617 1167 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260811 02:56:05.120834 1167 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786416707705494-12843-0/minicluster-data/ts-1/data,/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786416707705494-12843-0/minicluster-data/ts-1/wal
uuid: "63c7ec3943ee4c3987785755411738fa"
format_stamp: "Formatted at 2026-08-11 02:56:05 on dist-test-slave-9gcw"
server_key: "dda92a3d92854c2e7ff86e891533128a"
server_key_iv: "a2470fe6845fbbd4e72941b40f215dc7"
server_key_version: "encryptionkey@0"
I20260811 02:56:05.120988 1167 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786416707705494-12843-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786416707705494-12843-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786416707705494-12843-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 02:56:05.138343 1167 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 02:56:05.139134 1167 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 02:56:05.139386 1167 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 02:56:05.140053 1167 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 02:56:05.141232 1167 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 02:56:05.141309 1167 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 02:56:05.141388 1167 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 02:56:05.141446 1167 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 02:56:05.152068 1167 rpc_server.cc:307] RPC server started. Bound to: 127.12.138.194:44839
I20260811 02:56:05.152096 1312 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.138.194:44839 every 8 connection(s)
I20260811 02:56:05.153333 1167 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786416707705494-12843-0/minicluster-data/ts-1/data/info.pb
I20260811 02:56:05.155156 12843 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDawEwS/build/debug/bin/kudu as pid 1167
I20260811 02:56:05.155333 12843 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786416707705494-12843-0/minicluster-data/ts-1/wal/instance
I20260811 02:56:05.155679 12843 external_mini_cluster.cc:1428] Setting key f7830017b8af660455d244a33f1938a0
I20260811 02:56:05.158311 12843 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDawEwS/build/debug/bin/kudu
/tmp/dist-test-taskDawEwS/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786416707705494-12843-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786416707705494-12843-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786416707705494-12843-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786416707705494-12843-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.138.195:0
--local_ip_for_outbound_sockets=127.12.138.195
--webserver_interface=127.12.138.195
--webserver_port=0
--tserver_master_addrs=127.12.138.254:41141
--builtin_ntp_servers=127.12.138.212:35353
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260811 02:56:05.164958 994 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:56:05.155517 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.138.194:45367 (local address 127.12.138.254:41141)
0811 02:56:05.155687 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0811 02:56:05.155692 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 02:56:05.157066 (+ 1374us) server_negotiation.cc:407] Connection header received
0811 02:56:05.158962 (+ 1896us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:56:05.158965 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:56:05.159010 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:56:05.159088 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:56:05.160384 (+ 1296us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:56:05.161303 (+ 919us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:56:05.162487 (+ 1184us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:56:05.162929 (+ 442us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:56:05.163087 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:56:05.164274 (+ 1187us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:56:05.164281 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:56:05.164288 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 02:56:05.164310 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:56:05.164342 (+ 32us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 02:56:05.164359 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:56:05.164452 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0811 02:56:05.164625 (+ 173us) server_negotiation.cc:299] Negotiation successful
0811 02:56:05.164727 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260811 02:56:05.165732 1315 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:56:05.156141 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.138.254:41141 (local address 127.12.138.194:45367)
0811 02:56:05.156881 (+ 740us) negotiation.cc:107] Waiting for socket to connect
0811 02:56:05.156934 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0811 02:56:05.158695 (+ 1761us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:56:05.159315 (+ 620us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:56:05.159327 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:56:05.159439 (+ 112us) client_negotiation.cc:189] Negotiated authn=SASL
0811 02:56:05.160161 (+ 722us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:56:05.160182 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:56:05.161440 (+ 1258us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:56:05.161443 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:56:05.162345 (+ 902us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:56:05.162355 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:56:05.163092 (+ 737us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:56:05.163094 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:56:05.163271 (+ 177us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:56:05.163990 (+ 719us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 02:56:05.163991 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0811 02:56:05.164034 (+ 43us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 02:56:05.164099 (+ 65us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 02:56:05.164466 (+ 367us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 02:56:05.164472 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 02:56:05.164477 (+ 5us) client_negotiation.cc:769] Sending connection context
0811 02:56:05.164574 (+ 97us) client_negotiation.cc:240] Negotiation successful
0811 02:56:05.164816 (+ 242us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":288,"thread_start_us":153,"threads_started":1}
I20260811 02:56:05.167120 1313 heartbeater.cc:347] Connected to a master server at 127.12.138.254:41141
I20260811 02:56:05.167414 1313 heartbeater.cc:464] Registering TS with master...
I20260811 02:56:05.168140 1313 heartbeater.cc:511] Master 127.12.138.254:41141 requested a full tablet report, sending...
I20260811 02:56:05.169497 953 ts_manager.cc:194] Registered new tserver with Master: 63c7ec3943ee4c3987785755411738fa (127.12.138.194:44839)
I20260811 02:56:05.170181 953 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.12.138.194:45367
W20260811 02:56:05.278617 1316 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 02:56:05.278924 1316 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 02:56:05.278965 1316 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 02:56:05.283213 1316 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 02:56:05.283284 1316 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 02:56:05.283341 1316 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.138.195
I20260811 02:56:05.288657 1316 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.138.212:35353
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786416707705494-12843-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786416707705494-12843-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.138.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786416707705494-12843-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.12.138.195
--webserver_port=0
--tserver_master_addrs=127.12.138.254:41141
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1316
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.138.195
--log_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786416707705494-12843-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 02:09:49 UTC on ccycloud.quasar-fsnqqc.root.comops.site
build id 4579570
I20260811 02:56:05.289901 1316 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 02:56:05.291363 1316 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 02:56:05.301460 1322 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:56:05.301465 1325 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:56:05.301465 1316 server_base.cc:1034] running on GCE node
W20260811 02:56:05.301483 1323 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:56:05.302209 1316 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 02:56:05.302834 1316 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 02:56:05.303994 1316 hybrid_clock.cc:648] HybridClock initialized: now 1786416965303960 us; error 46 us; skew 500 ppm
I20260811 02:56:05.305939 1316 webserver.cc:460] Webserver started at http://127.12.138.195:41769/ using document root <none> and password file <none>
I20260811 02:56:05.306547 1316 fs_manager.cc:359] Metadata directory not provided
I20260811 02:56:05.306612 1316 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 02:56:05.306856 1316 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 02:56:05.309051 1316 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786416707705494-12843-0/minicluster-data/ts-2/data/instance:
uuid: "74f19c7383d448b6a4c074f164766dab"
format_stamp: "Formatted at 2026-08-11 02:56:05 on dist-test-slave-9gcw"
server_key: "a9a4024968225ef00e11a1a5a503e245"
server_key_iv: "88ab6ed38e0fbebf9b38b6e73287fbde"
server_key_version: "encryptionkey@0"
I20260811 02:56:05.309717 1316 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786416707705494-12843-0/minicluster-data/ts-2/wal/instance:
uuid: "74f19c7383d448b6a4c074f164766dab"
format_stamp: "Formatted at 2026-08-11 02:56:05 on dist-test-slave-9gcw"
server_key: "a9a4024968225ef00e11a1a5a503e245"
server_key_iv: "88ab6ed38e0fbebf9b38b6e73287fbde"
server_key_version: "encryptionkey@0"
I20260811 02:56:05.313791 1316 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260811 02:56:05.316521 1332 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 02:56:05.317984 1316 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 02:56:05.318082 1316 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786416707705494-12843-0/minicluster-data/ts-2/data,/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786416707705494-12843-0/minicluster-data/ts-2/wal
uuid: "74f19c7383d448b6a4c074f164766dab"
format_stamp: "Formatted at 2026-08-11 02:56:05 on dist-test-slave-9gcw"
server_key: "a9a4024968225ef00e11a1a5a503e245"
server_key_iv: "88ab6ed38e0fbebf9b38b6e73287fbde"
server_key_version: "encryptionkey@0"
I20260811 02:56:05.318169 1316 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786416707705494-12843-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786416707705494-12843-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786416707705494-12843-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 02:56:05.343912 1316 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 02:56:05.344586 1316 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 02:56:05.344755 1316 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 02:56:05.345328 1316 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 02:56:05.346639 1316 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 02:56:05.346698 1316 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 02:56:05.346766 1316 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 02:56:05.346802 1316 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 02:56:05.356475 1316 rpc_server.cc:307] RPC server started. Bound to: 127.12.138.195:35507
I20260811 02:56:05.356513 1458 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.138.195:35507 every 8 connection(s)
I20260811 02:56:05.357517 1316 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786416707705494-12843-0/minicluster-data/ts-2/data/info.pb
I20260811 02:56:05.367120 12843 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDawEwS/build/debug/bin/kudu as pid 1316
I20260811 02:56:05.367273 12843 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786416707705494-12843-0/minicluster-data/ts-2/wal/instance
I20260811 02:56:05.367651 12843 external_mini_cluster.cc:1428] Setting key 838e2863420874da243b8b8f8f29c86f
I20260811 02:56:05.368080 994 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:56:05.359626 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.138.195:58717 (local address 127.12.138.254:41141)
0811 02:56:05.359842 (+ 216us) server_negotiation.cc:206] Beginning negotiation
0811 02:56:05.359846 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 02:56:05.360936 (+ 1090us) server_negotiation.cc:407] Connection header received
0811 02:56:05.362488 (+ 1552us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:56:05.362491 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:56:05.362541 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:56:05.362623 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:56:05.363705 (+ 1082us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:56:05.364266 (+ 561us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:56:05.365712 (+ 1446us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:56:05.365932 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:56:05.366009 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:56:05.367484 (+ 1475us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:56:05.367489 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:56:05.367492 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 02:56:05.367509 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:56:05.367535 (+ 26us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 02:56:05.367546 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:56:05.367628 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0811 02:56:05.367851 (+ 223us) server_negotiation.cc:299] Negotiation successful
0811 02:56:05.367921 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":119}
I20260811 02:56:05.369259 1461 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:56:05.359996 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.138.254:41141 (local address 127.12.138.195:58717)
0811 02:56:05.360771 (+ 775us) negotiation.cc:107] Waiting for socket to connect
0811 02:56:05.360818 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0811 02:56:05.362254 (+ 1436us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:56:05.362893 (+ 639us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:56:05.362901 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:56:05.362999 (+ 98us) client_negotiation.cc:189] Negotiated authn=SASL
0811 02:56:05.363557 (+ 558us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:56:05.363571 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:56:05.364402 (+ 831us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:56:05.364406 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:56:05.365581 (+ 1175us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:56:05.365592 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:56:05.366087 (+ 495us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:56:05.366090 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:56:05.366270 (+ 180us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:56:05.367129 (+ 859us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 02:56:05.367131 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0811 02:56:05.367177 (+ 46us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 02:56:05.367278 (+ 101us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 02:56:05.367681 (+ 403us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 02:56:05.367690 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 02:56:05.367695 (+ 5us) client_negotiation.cc:769] Sending connection context
0811 02:56:05.367841 (+ 146us) client_negotiation.cc:240] Negotiation successful
0811 02:56:05.368145 (+ 304us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":300,"thread_start_us":123,"threads_started":1}
I20260811 02:56:05.370507 1459 heartbeater.cc:347] Connected to a master server at 127.12.138.254:41141
I20260811 02:56:05.370818 1459 heartbeater.cc:464] Registering TS with master...
I20260811 02:56:05.371369 1459 heartbeater.cc:511] Master 127.12.138.254:41141 requested a full tablet report, sending...
I20260811 02:56:05.372475 953 ts_manager.cc:194] Registered new tserver with Master: 74f19c7383d448b6a4c074f164766dab (127.12.138.195:35507)
I20260811 02:56:05.373064 953 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.12.138.195:58717
I20260811 02:56:05.382200 12843 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 02:56:05.388922 994 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:56:05.384284 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36804 (local address 127.12.138.254:41141)
0811 02:56:05.384464 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0811 02:56:05.384470 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 02:56:05.384712 (+ 242us) server_negotiation.cc:407] Connection header received
0811 02:56:05.384825 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:56:05.384830 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:56:05.384887 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:56:05.384985 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:56:05.385414 (+ 429us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:56:05.385981 (+ 567us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:56:05.387100 (+ 1119us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:56:05.387340 (+ 240us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:56:05.387460 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:56:05.388430 (+ 970us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:56:05.388435 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:56:05.388438 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 02:56:05.388460 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:56:05.388481 (+ 21us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 02:56:05.388491 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:56:05.388575 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0811 02:56:05.388688 (+ 113us) server_negotiation.cc:299] Negotiation successful
0811 02:56:05.388747 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260811 02:56:05.391978 953 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:36804:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 02:56:05.395102 953 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 02:56:05.410286 1475 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:56:05.405074 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.138.193:39445 (local address 127.0.0.1:57488)
0811 02:56:05.405454 (+ 380us) negotiation.cc:107] Waiting for socket to connect
0811 02:56:05.405468 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 02:56:05.405561 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:56:05.406252 (+ 691us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:56:05.406255 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:56:05.406267 (+ 12us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 02:56:05.406376 (+ 109us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:56:05.406386 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:56:05.407373 (+ 987us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:56:05.407376 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:56:05.408562 (+ 1186us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:56:05.408572 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:56:05.409645 (+ 1073us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:56:05.409649 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:56:05.409823 (+ 174us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:56:05.409872 (+ 49us) client_negotiation.cc:769] Sending connection context
0811 02:56:05.409981 (+ 109us) client_negotiation.cc:240] Negotiation successful
0811 02:56:05.410108 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":312,"mutex_wait_us":28,"thread_start_us":122,"threads_started":1}
I20260811 02:56:05.410333 1476 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:56:05.405515 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57488 (local address 127.12.138.193:39445)
0811 02:56:05.405857 (+ 342us) server_negotiation.cc:206] Beginning negotiation
0811 02:56:05.405863 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 02:56:05.405881 (+ 18us) server_negotiation.cc:407] Connection header received
0811 02:56:05.405938 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:56:05.405941 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:56:05.406098 (+ 157us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:56:05.406266 (+ 168us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 02:56:05.406519 (+ 253us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:56:05.407246 (+ 727us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:56:05.408698 (+ 1452us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:56:05.409500 (+ 802us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:56:05.409638 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:56:05.409900 (+ 262us) server_negotiation.cc:1035] Waiting for connection context
0811 02:56:05.410029 (+ 129us) server_negotiation.cc:299] Negotiation successful
0811 02:56:05.410140 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":247,"thread_start_us":71,"threads_started":1}
I20260811 02:56:05.410566 1471 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:56:05.403965 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.138.194:44839 (local address 127.0.0.1:37752)
0811 02:56:05.404794 (+ 829us) negotiation.cc:107] Waiting for socket to connect
0811 02:56:05.404848 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0811 02:56:05.404982 (+ 134us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:56:05.405801 (+ 819us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:56:05.405807 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:56:05.405829 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 02:56:05.405992 (+ 163us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:56:05.406000 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:56:05.407518 (+ 1518us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:56:05.407521 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:56:05.408830 (+ 1309us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:56:05.408841 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:56:05.410109 (+ 1268us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:56:05.410113 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:56:05.410209 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:56:05.410248 (+ 39us) client_negotiation.cc:769] Sending connection context
0811 02:56:05.410323 (+ 75us) client_negotiation.cc:240] Negotiation successful
0811 02:56:05.410408 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":646,"mutex_wait_us":34,"thread_start_us":87,"threads_started":1}
I20260811 02:56:05.411108 1473 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:56:05.404075 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37752 (local address 127.12.138.194:44839)
0811 02:56:05.404944 (+ 869us) server_negotiation.cc:206] Beginning negotiation
0811 02:56:05.404956 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 02:56:05.404981 (+ 25us) server_negotiation.cc:407] Connection header received
0811 02:56:05.405233 (+ 252us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:56:05.405239 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:56:05.405452 (+ 213us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:56:05.405617 (+ 165us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 02:56:05.406133 (+ 516us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:56:05.407378 (+ 1245us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:56:05.408969 (+ 1591us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:56:05.409767 (+ 798us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:56:05.409885 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:56:05.410042 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0811 02:56:05.410835 (+ 793us) server_negotiation.cc:299] Negotiation successful
0811 02:56:05.410932 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":761,"thread_start_us":141,"threads_started":1}
I20260811 02:56:05.411818 1472 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:56:05.404042 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.138.195:35507 (local address 127.0.0.1:56730)
0811 02:56:05.405119 (+ 1077us) negotiation.cc:107] Waiting for socket to connect
0811 02:56:05.405133 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 02:56:05.405520 (+ 387us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:56:05.406165 (+ 645us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:56:05.406167 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:56:05.406182 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 02:56:05.406346 (+ 164us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:56:05.406355 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:56:05.407816 (+ 1461us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:56:05.407819 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:56:05.409001 (+ 1182us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:56:05.409017 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:56:05.410218 (+ 1201us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:56:05.410222 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:56:05.410332 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:56:05.410367 (+ 35us) client_negotiation.cc:769] Sending connection context
0811 02:56:05.411596 (+ 1229us) client_negotiation.cc:240] Negotiation successful
0811 02:56:05.411686 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1008,"mutex_wait_us":26,"thread_start_us":108,"threads_started":1}
I20260811 02:56:05.412065 1474 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:56:05.404238 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56730 (local address 127.12.138.195:35507)
0811 02:56:05.404662 (+ 424us) server_negotiation.cc:206] Beginning negotiation
0811 02:56:05.404679 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0811 02:56:05.405245 (+ 566us) server_negotiation.cc:407] Connection header received
0811 02:56:05.405677 (+ 432us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:56:05.405681 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:56:05.405864 (+ 183us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:56:05.405985 (+ 121us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 02:56:05.406457 (+ 472us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:56:05.407692 (+ 1235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:56:05.409128 (+ 1436us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:56:05.409852 (+ 724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:56:05.409984 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:56:05.410154 (+ 170us) server_negotiation.cc:1035] Waiting for connection context
0811 02:56:05.411627 (+ 1473us) server_negotiation.cc:299] Negotiation successful
0811 02:56:05.411722 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":317,"thread_start_us":106,"threads_started":1}
I20260811 02:56:05.413913 1239 tablet_service.cc:1467] Processing CreateTablet for tablet 70b43800f19c4f35a011f77141a4f923 (DEFAULT_TABLE table=test-table [id=709af6d7292a428fa1f1aac976c1e3f0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 02:56:05.413914 1090 tablet_service.cc:1467] Processing CreateTablet for tablet 70b43800f19c4f35a011f77141a4f923 (DEFAULT_TABLE table=test-table [id=709af6d7292a428fa1f1aac976c1e3f0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 02:56:05.415122 1090 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 70b43800f19c4f35a011f77141a4f923. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 02:56:05.415122 1239 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 70b43800f19c4f35a011f77141a4f923. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 02:56:05.415784 1386 tablet_service.cc:1467] Processing CreateTablet for tablet 70b43800f19c4f35a011f77141a4f923 (DEFAULT_TABLE table=test-table [id=709af6d7292a428fa1f1aac976c1e3f0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 02:56:05.416695 1386 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 70b43800f19c4f35a011f77141a4f923. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 02:56:05.423411 1477 tablet_bootstrap.cc:492] T 70b43800f19c4f35a011f77141a4f923 P 63c7ec3943ee4c3987785755411738fa: Bootstrap starting.
I20260811 02:56:05.423887 1479 tablet_bootstrap.cc:492] T 70b43800f19c4f35a011f77141a4f923 P f3862759323e4abe84a20f26e81a3c8a: Bootstrap starting.
I20260811 02:56:05.425515 1481 tablet_bootstrap.cc:492] T 70b43800f19c4f35a011f77141a4f923 P 74f19c7383d448b6a4c074f164766dab: Bootstrap starting.
I20260811 02:56:05.426082 1479 tablet_bootstrap.cc:654] T 70b43800f19c4f35a011f77141a4f923 P f3862759323e4abe84a20f26e81a3c8a: Neither blocks nor log segments found. Creating new log.
I20260811 02:56:05.426950 1479 log.cc:824] T 70b43800f19c4f35a011f77141a4f923 P f3862759323e4abe84a20f26e81a3c8a: Log is configured to *not* fsync() on all Append() calls
I20260811 02:56:05.427459 1481 tablet_bootstrap.cc:654] T 70b43800f19c4f35a011f77141a4f923 P 74f19c7383d448b6a4c074f164766dab: Neither blocks nor log segments found. Creating new log.
I20260811 02:56:05.428319 1481 log.cc:824] T 70b43800f19c4f35a011f77141a4f923 P 74f19c7383d448b6a4c074f164766dab: Log is configured to *not* fsync() on all Append() calls
I20260811 02:56:05.429384 1479 tablet_bootstrap.cc:492] T 70b43800f19c4f35a011f77141a4f923 P f3862759323e4abe84a20f26e81a3c8a: No bootstrap required, opened a new log
I20260811 02:56:05.429374 1477 tablet_bootstrap.cc:654] T 70b43800f19c4f35a011f77141a4f923 P 63c7ec3943ee4c3987785755411738fa: Neither blocks nor log segments found. Creating new log.
I20260811 02:56:05.429587 1479 ts_tablet_manager.cc:1397] T 70b43800f19c4f35a011f77141a4f923 P f3862759323e4abe84a20f26e81a3c8a: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260811 02:56:05.430243 1481 tablet_bootstrap.cc:492] T 70b43800f19c4f35a011f77141a4f923 P 74f19c7383d448b6a4c074f164766dab: No bootstrap required, opened a new log
I20260811 02:56:05.430215 1477 log.cc:824] T 70b43800f19c4f35a011f77141a4f923 P 63c7ec3943ee4c3987785755411738fa: Log is configured to *not* fsync() on all Append() calls
I20260811 02:56:05.430454 1481 ts_tablet_manager.cc:1397] T 70b43800f19c4f35a011f77141a4f923 P 74f19c7383d448b6a4c074f164766dab: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260811 02:56:05.432808 1477 tablet_bootstrap.cc:492] T 70b43800f19c4f35a011f77141a4f923 P 63c7ec3943ee4c3987785755411738fa: No bootstrap required, opened a new log
I20260811 02:56:05.433003 1477 ts_tablet_manager.cc:1397] T 70b43800f19c4f35a011f77141a4f923 P 63c7ec3943ee4c3987785755411738fa: Time spent bootstrapping tablet: real 0.010s user 0.004s sys 0.000s
I20260811 02:56:05.433844 1481 raft_consensus.cc:348] T 70b43800f19c4f35a011f77141a4f923 P 74f19c7383d448b6a4c074f164766dab [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "63c7ec3943ee4c3987785755411738fa" member_type: VOTER last_known_addr { host: "127.12.138.194" port: 44839 } } peers { permanent_uuid: "74f19c7383d448b6a4c074f164766dab" member_type: VOTER last_known_addr { host: "127.12.138.195" port: 35507 } } peers { permanent_uuid: "f3862759323e4abe84a20f26e81a3c8a" member_type: VOTER last_known_addr { host: "127.12.138.193" port: 39445 } }
I20260811 02:56:05.434095 1481 raft_consensus.cc:374] T 70b43800f19c4f35a011f77141a4f923 P 74f19c7383d448b6a4c074f164766dab [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 02:56:05.434158 1481 raft_consensus.cc:729] T 70b43800f19c4f35a011f77141a4f923 P 74f19c7383d448b6a4c074f164766dab [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 74f19c7383d448b6a4c074f164766dab, State: Initialized, Role: FOLLOWER
I20260811 02:56:05.434031 1479 raft_consensus.cc:348] T 70b43800f19c4f35a011f77141a4f923 P f3862759323e4abe84a20f26e81a3c8a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "63c7ec3943ee4c3987785755411738fa" member_type: VOTER last_known_addr { host: "127.12.138.194" port: 44839 } } peers { permanent_uuid: "74f19c7383d448b6a4c074f164766dab" member_type: VOTER last_known_addr { host: "127.12.138.195" port: 35507 } } peers { permanent_uuid: "f3862759323e4abe84a20f26e81a3c8a" member_type: VOTER last_known_addr { host: "127.12.138.193" port: 39445 } }
I20260811 02:56:05.434296 1479 raft_consensus.cc:374] T 70b43800f19c4f35a011f77141a4f923 P f3862759323e4abe84a20f26e81a3c8a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 02:56:05.434351 1479 raft_consensus.cc:729] T 70b43800f19c4f35a011f77141a4f923 P f3862759323e4abe84a20f26e81a3c8a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f3862759323e4abe84a20f26e81a3c8a, State: Initialized, Role: FOLLOWER
I20260811 02:56:05.434679 1481 consensus_queue.cc:260] T 70b43800f19c4f35a011f77141a4f923 P 74f19c7383d448b6a4c074f164766dab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "63c7ec3943ee4c3987785755411738fa" member_type: VOTER last_known_addr { host: "127.12.138.194" port: 44839 } } peers { permanent_uuid: "74f19c7383d448b6a4c074f164766dab" member_type: VOTER last_known_addr { host: "127.12.138.195" port: 35507 } } peers { permanent_uuid: "f3862759323e4abe84a20f26e81a3c8a" member_type: VOTER last_known_addr { host: "127.12.138.193" port: 39445 } }
I20260811 02:56:05.434768 1479 consensus_queue.cc:260] T 70b43800f19c4f35a011f77141a4f923 P f3862759323e4abe84a20f26e81a3c8a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "63c7ec3943ee4c3987785755411738fa" member_type: VOTER last_known_addr { host: "127.12.138.194" port: 44839 } } peers { permanent_uuid: "74f19c7383d448b6a4c074f164766dab" member_type: VOTER last_known_addr { host: "127.12.138.195" port: 35507 } } peers { permanent_uuid: "f3862759323e4abe84a20f26e81a3c8a" member_type: VOTER last_known_addr { host: "127.12.138.193" port: 39445 } }
I20260811 02:56:05.435515 1164 heartbeater.cc:503] Master 127.12.138.254:41141 was elected leader, sending a full tablet report...
I20260811 02:56:05.435837 1479 ts_tablet_manager.cc:1428] T 70b43800f19c4f35a011f77141a4f923 P f3862759323e4abe84a20f26e81a3c8a: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260811 02:56:05.436650 1481 ts_tablet_manager.cc:1428] T 70b43800f19c4f35a011f77141a4f923 P 74f19c7383d448b6a4c074f164766dab: Time spent starting tablet: real 0.006s user 0.001s sys 0.004s
I20260811 02:56:05.436779 1477 raft_consensus.cc:348] T 70b43800f19c4f35a011f77141a4f923 P 63c7ec3943ee4c3987785755411738fa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "63c7ec3943ee4c3987785755411738fa" member_type: VOTER last_known_addr { host: "127.12.138.194" port: 44839 } } peers { permanent_uuid: "74f19c7383d448b6a4c074f164766dab" member_type: VOTER last_known_addr { host: "127.12.138.195" port: 35507 } } peers { permanent_uuid: "f3862759323e4abe84a20f26e81a3c8a" member_type: VOTER last_known_addr { host: "127.12.138.193" port: 39445 } }
I20260811 02:56:05.437023 1459 heartbeater.cc:503] Master 127.12.138.254:41141 was elected leader, sending a full tablet report...
I20260811 02:56:05.437058 1477 raft_consensus.cc:374] T 70b43800f19c4f35a011f77141a4f923 P 63c7ec3943ee4c3987785755411738fa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 02:56:05.437124 1477 raft_consensus.cc:729] T 70b43800f19c4f35a011f77141a4f923 P 63c7ec3943ee4c3987785755411738fa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 63c7ec3943ee4c3987785755411738fa, State: Initialized, Role: FOLLOWER
I20260811 02:56:05.437567 1477 consensus_queue.cc:260] T 70b43800f19c4f35a011f77141a4f923 P 63c7ec3943ee4c3987785755411738fa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "63c7ec3943ee4c3987785755411738fa" member_type: VOTER last_known_addr { host: "127.12.138.194" port: 44839 } } peers { permanent_uuid: "74f19c7383d448b6a4c074f164766dab" member_type: VOTER last_known_addr { host: "127.12.138.195" port: 35507 } } peers { permanent_uuid: "f3862759323e4abe84a20f26e81a3c8a" member_type: VOTER last_known_addr { host: "127.12.138.193" port: 39445 } }
I20260811 02:56:05.438328 1313 heartbeater.cc:503] Master 127.12.138.254:41141 was elected leader, sending a full tablet report...
I20260811 02:56:05.438535 1477 ts_tablet_manager.cc:1428] T 70b43800f19c4f35a011f77141a4f923 P 63c7ec3943ee4c3987785755411738fa: Time spent starting tablet: real 0.005s user 0.003s sys 0.004s
W20260811 02:56:05.454023 1165 tablet.cc:2369] T 70b43800f19c4f35a011f77141a4f923 P f3862759323e4abe84a20f26e81a3c8a: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 02:56:05.511243 1486 raft_consensus.cc:482] T 70b43800f19c4f35a011f77141a4f923 P 74f19c7383d448b6a4c074f164766dab [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 02:56:05.511493 1486 raft_consensus.cc:504] T 70b43800f19c4f35a011f77141a4f923 P 74f19c7383d448b6a4c074f164766dab [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "63c7ec3943ee4c3987785755411738fa" member_type: VOTER last_known_addr { host: "127.12.138.194" port: 44839 } } peers { permanent_uuid: "74f19c7383d448b6a4c074f164766dab" member_type: VOTER last_known_addr { host: "127.12.138.195" port: 35507 } } peers { permanent_uuid: "f3862759323e4abe84a20f26e81a3c8a" member_type: VOTER last_known_addr { host: "127.12.138.193" port: 39445 } }
I20260811 02:56:05.512912 1486 leader_election.cc:288] T 70b43800f19c4f35a011f77141a4f923 P 74f19c7383d448b6a4c074f164766dab [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 63c7ec3943ee4c3987785755411738fa (127.12.138.194:44839), f3862759323e4abe84a20f26e81a3c8a (127.12.138.193:39445)
I20260811 02:56:05.517376 1473 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:56:05.513269 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.138.195:40235 (local address 127.12.138.194:44839)
0811 02:56:05.513416 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0811 02:56:05.513423 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 02:56:05.513443 (+ 20us) server_negotiation.cc:407] Connection header received
0811 02:56:05.513509 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:56:05.513512 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:56:05.513573 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:56:05.513669 (+ 96us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 02:56:05.514198 (+ 529us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:56:05.514913 (+ 715us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:56:05.516148 (+ 1235us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:56:05.516611 (+ 463us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:56:05.516710 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:56:05.516758 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0811 02:56:05.517119 (+ 361us) server_negotiation.cc:299] Negotiation successful
0811 02:56:05.517239 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260811 02:56:05.517378 1461 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:56:05.513093 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.138.194:44839 (local address 127.12.138.195:40235)
0811 02:56:05.513282 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0811 02:56:05.513300 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 02:56:05.513409 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:56:05.513798 (+ 389us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:56:05.513802 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:56:05.513828 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 02:56:05.513959 (+ 131us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:56:05.513967 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:56:05.515054 (+ 1087us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:56:05.515065 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:56:05.515996 (+ 931us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:56:05.516012 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:56:05.516850 (+ 838us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:56:05.516855 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:56:05.516963 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:56:05.516994 (+ 31us) client_negotiation.cc:769] Sending connection context
0811 02:56:05.517113 (+ 119us) client_negotiation.cc:240] Negotiation successful
0811 02:56:05.517220 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":82}
I20260811 02:56:05.517607 1491 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:56:05.513303 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.138.193:39445 (local address 127.12.138.195:45161)
0811 02:56:05.513691 (+ 388us) negotiation.cc:107] Waiting for socket to connect
0811 02:56:05.513708 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 02:56:05.513879 (+ 171us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:56:05.514204 (+ 325us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:56:05.514206 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:56:05.514226 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 02:56:05.514388 (+ 162us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:56:05.514395 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:56:05.515359 (+ 964us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:56:05.515362 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:56:05.516179 (+ 817us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:56:05.516189 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:56:05.517132 (+ 943us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:56:05.517136 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:56:05.517265 (+ 129us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:56:05.517289 (+ 24us) client_negotiation.cc:769] Sending connection context
0811 02:56:05.517371 (+ 82us) client_negotiation.cc:240] Negotiation successful
0811 02:56:05.517447 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":314,"spinlock_wait_cycles":896,"thread_start_us":153,"threads_started":1}
I20260811 02:56:05.517658 1476 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:56:05.513503 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.138.195:45161 (local address 127.12.138.193:39445)
0811 02:56:05.513719 (+ 216us) server_negotiation.cc:206] Beginning negotiation
0811 02:56:05.513724 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 02:56:05.513802 (+ 78us) server_negotiation.cc:407] Connection header received
0811 02:56:05.514016 (+ 214us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:56:05.514021 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:56:05.514087 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:56:05.514170 (+ 83us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 02:56:05.514547 (+ 377us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:56:05.515208 (+ 661us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:56:05.516326 (+ 1118us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:56:05.517006 (+ 680us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:56:05.517130 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:56:05.517205 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0811 02:56:05.517417 (+ 212us) server_negotiation.cc:299] Negotiation successful
0811 02:56:05.517500 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":116}
I20260811 02:56:05.518230 1261 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "70b43800f19c4f35a011f77141a4f923" candidate_uuid: "74f19c7383d448b6a4c074f164766dab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "63c7ec3943ee4c3987785755411738fa" is_pre_election: true
I20260811 02:56:05.518239 1112 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "70b43800f19c4f35a011f77141a4f923" candidate_uuid: "74f19c7383d448b6a4c074f164766dab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f3862759323e4abe84a20f26e81a3c8a" is_pre_election: true
I20260811 02:56:05.518532 1261 raft_consensus.cc:2454] T 70b43800f19c4f35a011f77141a4f923 P 63c7ec3943ee4c3987785755411738fa [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 74f19c7383d448b6a4c074f164766dab in term 0.
I20260811 02:56:05.518563 1112 raft_consensus.cc:2454] T 70b43800f19c4f35a011f77141a4f923 P f3862759323e4abe84a20f26e81a3c8a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 74f19c7383d448b6a4c074f164766dab in term 0.
I20260811 02:56:05.519093 1333 leader_election.cc:302] T 70b43800f19c4f35a011f77141a4f923 P 74f19c7383d448b6a4c074f164766dab [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 63c7ec3943ee4c3987785755411738fa, 74f19c7383d448b6a4c074f164766dab, f3862759323e4abe84a20f26e81a3c8a; no voters:
I20260811 02:56:05.519421 1486 raft_consensus.cc:2781] T 70b43800f19c4f35a011f77141a4f923 P 74f19c7383d448b6a4c074f164766dab [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 02:56:05.519500 1486 raft_consensus.cc:482] T 70b43800f19c4f35a011f77141a4f923 P 74f19c7383d448b6a4c074f164766dab [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 02:56:05.519531 1486 raft_consensus.cc:3037] T 70b43800f19c4f35a011f77141a4f923 P 74f19c7383d448b6a4c074f164766dab [term 0 FOLLOWER]: Advancing to term 1
I20260811 02:56:05.520607 1486 raft_consensus.cc:504] T 70b43800f19c4f35a011f77141a4f923 P 74f19c7383d448b6a4c074f164766dab [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "63c7ec3943ee4c3987785755411738fa" member_type: VOTER last_known_addr { host: "127.12.138.194" port: 44839 } } peers { permanent_uuid: "74f19c7383d448b6a4c074f164766dab" member_type: VOTER last_known_addr { host: "127.12.138.195" port: 35507 } } peers { permanent_uuid: "f3862759323e4abe84a20f26e81a3c8a" member_type: VOTER last_known_addr { host: "127.12.138.193" port: 39445 } }
I20260811 02:56:05.521018 1486 leader_election.cc:288] T 70b43800f19c4f35a011f77141a4f923 P 74f19c7383d448b6a4c074f164766dab [CANDIDATE]: Term 1 election: Requested vote from peers 63c7ec3943ee4c3987785755411738fa (127.12.138.194:44839), f3862759323e4abe84a20f26e81a3c8a (127.12.138.193:39445)
I20260811 02:56:05.521416 1261 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "70b43800f19c4f35a011f77141a4f923" candidate_uuid: "74f19c7383d448b6a4c074f164766dab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "63c7ec3943ee4c3987785755411738fa"
I20260811 02:56:05.521528 1261 raft_consensus.cc:3037] T 70b43800f19c4f35a011f77141a4f923 P 63c7ec3943ee4c3987785755411738fa [term 0 FOLLOWER]: Advancing to term 1
I20260811 02:56:05.521518 1112 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "70b43800f19c4f35a011f77141a4f923" candidate_uuid: "74f19c7383d448b6a4c074f164766dab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f3862759323e4abe84a20f26e81a3c8a"
I20260811 02:56:05.521778 1112 raft_consensus.cc:3037] T 70b43800f19c4f35a011f77141a4f923 P f3862759323e4abe84a20f26e81a3c8a [term 0 FOLLOWER]: Advancing to term 1
I20260811 02:56:05.522802 1261 raft_consensus.cc:2454] T 70b43800f19c4f35a011f77141a4f923 P 63c7ec3943ee4c3987785755411738fa [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 74f19c7383d448b6a4c074f164766dab in term 1.
I20260811 02:56:05.523120 1334 leader_election.cc:302] T 70b43800f19c4f35a011f77141a4f923 P 74f19c7383d448b6a4c074f164766dab [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 63c7ec3943ee4c3987785755411738fa, 74f19c7383d448b6a4c074f164766dab; no voters:
I20260811 02:56:05.523113 1112 raft_consensus.cc:2454] T 70b43800f19c4f35a011f77141a4f923 P f3862759323e4abe84a20f26e81a3c8a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 74f19c7383d448b6a4c074f164766dab in term 1.
I20260811 02:56:05.523343 1486 raft_consensus.cc:2781] T 70b43800f19c4f35a011f77141a4f923 P 74f19c7383d448b6a4c074f164766dab [term 1 FOLLOWER]: Leader election won for term 1
I20260811 02:56:05.523640 1486 raft_consensus.cc:686] T 70b43800f19c4f35a011f77141a4f923 P 74f19c7383d448b6a4c074f164766dab [term 1 LEADER]: Becoming Leader. State: Replica: 74f19c7383d448b6a4c074f164766dab, State: Running, Role: LEADER
I20260811 02:56:05.524061 1486 consensus_queue.cc:237] T 70b43800f19c4f35a011f77141a4f923 P 74f19c7383d448b6a4c074f164766dab [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "63c7ec3943ee4c3987785755411738fa" member_type: VOTER last_known_addr { host: "127.12.138.194" port: 44839 } } peers { permanent_uuid: "74f19c7383d448b6a4c074f164766dab" member_type: VOTER last_known_addr { host: "127.12.138.195" port: 35507 } } peers { permanent_uuid: "f3862759323e4abe84a20f26e81a3c8a" member_type: VOTER last_known_addr { host: "127.12.138.193" port: 39445 } }
I20260811 02:56:05.527700 953 catalog_manager.cc:5495] T 70b43800f19c4f35a011f77141a4f923 P 74f19c7383d448b6a4c074f164766dab reported cstate change: term changed from 0 to 1, leader changed from <none> to 74f19c7383d448b6a4c074f164766dab (127.12.138.195). New cstate: current_term: 1 leader_uuid: "74f19c7383d448b6a4c074f164766dab" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "63c7ec3943ee4c3987785755411738fa" member_type: VOTER last_known_addr { host: "127.12.138.194" port: 44839 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "74f19c7383d448b6a4c074f164766dab" member_type: VOTER last_known_addr { host: "127.12.138.195" port: 35507 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f3862759323e4abe84a20f26e81a3c8a" member_type: VOTER last_known_addr { host: "127.12.138.193" port: 39445 } health_report { overall_health: UNKNOWN } } }
I20260811 02:56:05.566625 1474 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:56:05.563169 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56746 (local address 127.12.138.195:35507)
0811 02:56:05.563321 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0811 02:56:05.563325 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 02:56:05.563339 (+ 14us) server_negotiation.cc:407] Connection header received
0811 02:56:05.563420 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:56:05.563423 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:56:05.563480 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:56:05.563583 (+ 103us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 02:56:05.563829 (+ 246us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:56:05.564407 (+ 578us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:56:05.565238 (+ 831us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:56:05.565459 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:56:05.565542 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:56:05.565815 (+ 273us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 02:56:05.566267 (+ 452us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 02:56:05.566368 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0811 02:56:05.566439 (+ 71us) server_negotiation.cc:299] Negotiation successful
0811 02:56:05.566488 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260811 02:56:05.572868 1261 raft_consensus.cc:1261] T 70b43800f19c4f35a011f77141a4f923 P 63c7ec3943ee4c3987785755411738fa [term 1 FOLLOWER]: Refusing update from remote peer 74f19c7383d448b6a4c074f164766dab: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 02:56:05.572889 1112 raft_consensus.cc:1261] T 70b43800f19c4f35a011f77141a4f923 P f3862759323e4abe84a20f26e81a3c8a [term 1 FOLLOWER]: Refusing update from remote peer 74f19c7383d448b6a4c074f164766dab: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 02:56:05.573803 1492 consensus_queue.cc:1035] T 70b43800f19c4f35a011f77141a4f923 P 74f19c7383d448b6a4c074f164766dab [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3862759323e4abe84a20f26e81a3c8a" member_type: VOTER last_known_addr { host: "127.12.138.193" port: 39445 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 02:56:05.574052 1486 consensus_queue.cc:1035] T 70b43800f19c4f35a011f77141a4f923 P 74f19c7383d448b6a4c074f164766dab [LEADER]: Connected to new peer: Peer: permanent_uuid: "63c7ec3943ee4c3987785755411738fa" member_type: VOTER last_known_addr { host: "127.12.138.194" port: 44839 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 02:56:05.582007 1495 mvcc.cc:205] Tried to move back new op lower bound from 7317163890978500608 to 7317163890789572608. Current Snapshot: MvccSnapshot[applied={T|T < 7317163890978500608}]
I20260811 02:56:05.582712 1497 mvcc.cc:205] Tried to move back new op lower bound from 7317163890978500608 to 7317163890789572608. Current Snapshot: MvccSnapshot[applied={T|T < 7317163890978500608}]
I20260811 02:56:05.583685 1498 mvcc.cc:205] Tried to move back new op lower bound from 7317163890978500608 to 7317163890789572608. Current Snapshot: MvccSnapshot[applied={T|T < 7317163890978500608}]
I20260811 02:56:05.591670 953 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:36804:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 02:56:05.591847 953 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:36804:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 02:56:05.594623 953 catalog_manager.cc:5782] T 00000000000000000000000000000000 P f8ba95b215cc402aa2b89b6bf227168d: Sending DeleteTablet for 3 replicas of tablet 70b43800f19c4f35a011f77141a4f923
I20260811 02:56:05.595606 1239 tablet_service.cc:1514] Processing DeleteTablet for tablet 70b43800f19c4f35a011f77141a4f923 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 02:56:05 UTC) from {username='slave'} at 127.0.0.1:37752
I20260811 02:56:05.596204 1507 tablet_replica.cc:330] T 70b43800f19c4f35a011f77141a4f923 P 63c7ec3943ee4c3987785755411738fa: stopping tablet replica
I20260811 02:56:05.596170 1090 tablet_service.cc:1514] Processing DeleteTablet for tablet 70b43800f19c4f35a011f77141a4f923 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 02:56:05 UTC) from {username='slave'} at 127.0.0.1:57488
I20260811 02:56:05.596205 1386 tablet_service.cc:1514] Processing DeleteTablet for tablet 70b43800f19c4f35a011f77141a4f923 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 02:56:05 UTC) from {username='slave'} at 127.0.0.1:56730
I20260811 02:56:05.596639 1507 raft_consensus.cc:2229] T 70b43800f19c4f35a011f77141a4f923 P 63c7ec3943ee4c3987785755411738fa [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 02:56:05.596844 1510 tablet_replica.cc:330] T 70b43800f19c4f35a011f77141a4f923 P 74f19c7383d448b6a4c074f164766dab: stopping tablet replica
I20260811 02:56:05.597098 1509 tablet_replica.cc:330] T 70b43800f19c4f35a011f77141a4f923 P f3862759323e4abe84a20f26e81a3c8a: stopping tablet replica
I20260811 02:56:05.597133 1510 raft_consensus.cc:2229] T 70b43800f19c4f35a011f77141a4f923 P 74f19c7383d448b6a4c074f164766dab [term 1 LEADER]: Raft consensus shutting down.
I20260811 02:56:05.597167 1507 raft_consensus.cc:2258] T 70b43800f19c4f35a011f77141a4f923 P 63c7ec3943ee4c3987785755411738fa [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 02:56:05.597501 12843 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDawEwS/build/debug/bin/kudu with pid 1019
I20260811 02:56:05.597595 1510 raft_consensus.cc:2258] T 70b43800f19c4f35a011f77141a4f923 P 74f19c7383d448b6a4c074f164766dab [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 02:56:05.604629 1507 ts_tablet_manager.cc:1905] T 70b43800f19c4f35a011f77141a4f923 P 63c7ec3943ee4c3987785755411738fa: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 02:56:05.608217 1507 ts_tablet_manager.cc:1918] T 70b43800f19c4f35a011f77141a4f923 P 63c7ec3943ee4c3987785755411738fa: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 02:56:05.608403 1507 log.cc:1196] T 70b43800f19c4f35a011f77141a4f923 P 63c7ec3943ee4c3987785755411738fa: Deleting WAL directory at /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786416707705494-12843-0/minicluster-data/ts-1/wal/wals/70b43800f19c4f35a011f77141a4f923
I20260811 02:56:05.608855 1507 ts_tablet_manager.cc:1939] T 70b43800f19c4f35a011f77141a4f923 P 63c7ec3943ee4c3987785755411738fa: Deleting consensus metadata
I20260811 02:56:05.610379 935 catalog_manager.cc:4841] TS 63c7ec3943ee4c3987785755411738fa (127.12.138.194:44839): tablet 70b43800f19c4f35a011f77141a4f923 (table test-table [id=709af6d7292a428fa1f1aac976c1e3f0]) successfully deleted
I20260811 02:56:05.614055 1510 ts_tablet_manager.cc:1905] T 70b43800f19c4f35a011f77141a4f923 P 74f19c7383d448b6a4c074f164766dab: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260811 02:56:05.617020 934 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.12.138.193:39445 (error 108)
I20260811 02:56:05.617668 12843 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDawEwS/build/debug/bin/kudu with pid 1167
I20260811 02:56:05.617794 1510 ts_tablet_manager.cc:1918] T 70b43800f19c4f35a011f77141a4f923 P 74f19c7383d448b6a4c074f164766dab: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 02:56:05.617925 1510 log.cc:1196] T 70b43800f19c4f35a011f77141a4f923 P 74f19c7383d448b6a4c074f164766dab: Deleting WAL directory at /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786416707705494-12843-0/minicluster-data/ts-2/wal/wals/70b43800f19c4f35a011f77141a4f923
I20260811 02:56:05.618319 1510 ts_tablet_manager.cc:1939] T 70b43800f19c4f35a011f77141a4f923 P 74f19c7383d448b6a4c074f164766dab: Deleting consensus metadata
I20260811 02:56:05.619608 936 catalog_manager.cc:4841] TS 74f19c7383d448b6a4c074f164766dab (127.12.138.195:35507): tablet 70b43800f19c4f35a011f77141a4f923 (table test-table [id=709af6d7292a428fa1f1aac976c1e3f0]) successfully deleted
I20260811 02:56:05.623200 1471 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:56:05.622859 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.138.193:39445 (local address 127.0.0.1:57496)
0811 02:56:05.623018 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0811 02:56:05.623113 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.138.193:39445: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
W20260811 02:56:05.623430 934 catalog_manager.cc:4584] TS f3862759323e4abe84a20f26e81a3c8a (127.12.138.193:39445): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 70b43800f19c4f35a011f77141a4f923: Network error: Client connection negotiation failed: client connection to 127.12.138.193:39445: connect: Connection refused (error 111)
I20260811 02:56:05.630347 12843 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDawEwS/build/debug/bin/kudu with pid 1316
I20260811 02:56:05.640097 12843 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDawEwS/build/debug/bin/kudu with pid 917
2026-08-11T02:56:05Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/1 (1236 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/2
Loading random data
Initializing database '/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786416707705494-12843-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 02:56:05 dist-test-slave-9gcw krb5kdc[1514](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 02:56:05 dist-test-slave-9gcw krb5kdc[1514](info): set up 2 sockets
Aug 11 02:56:05 dist-test-slave-9gcw krb5kdc[1514](info): commencing operation
krb5kdc: starting...
W20260811 02:56:06.840147 12843 mini_kdc.cc:121] Time spent starting KDC: real 1.173s user 0.002s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786416707705494-12843-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786416707705494-12843-0/krb5kdc/test-user.keytab.
Aug 11 02:56:06 dist-test-slave-9gcw krb5kdc[1514](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416966, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T02:56:06Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T02:56:06Z Disabled control of system clock
WARNING: no policy specified for kudu/127.12.138.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.138.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.138.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786416707705494-12843-0/krb5kdc/kudu_127.12.138.254.keytab.
Entry for principal kudu/127.12.138.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786416707705494-12843-0/krb5kdc/kudu_127.12.138.254.keytab.
I20260811 02:56:06.985409 12843 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDawEwS/build/debug/bin/kudu
/tmp/dist-test-taskDawEwS/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786416707705494-12843-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786416707705494-12843-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786416707705494-12843-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786416707705494-12843-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.138.254:43779
--webserver_interface=127.12.138.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786416707705494-12843-0/krb5kdc/kudu_127.12.138.254.keytab
--principal=kudu/127.12.138.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.138.212:45985
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.12.138.254:43779
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786416707705494-12843-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786416707705494-12843-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786416707705494-12843-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 02:56:07.104902 1529 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 02:56:07.105271 1529 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 02:56:07.105365 1529 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 02:56:07.109411 1529 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 02:56:07.109469 1529 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 02:56:07.109489 1529 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 02:56:07.109506 1529 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 02:56:07.109522 1529 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 02:56:07.115051 1529 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.138.212:45985
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786416707705494-12843-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786416707705494-12843-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.12.138.254:43779
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786416707705494-12843-0/krb5kdc/kudu_127.12.138.254.keytab
--principal=kudu/127.12.138.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.138.254:43779
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786416707705494-12843-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.12.138.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1529
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786416707705494-12843-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 02:09:49 UTC on ccycloud.quasar-fsnqqc.root.comops.site
build id 4579570
I20260811 02:56:07.116199 1529 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 02:56:07.117292 1529 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 02:56:07.125941 1536 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:56:07.125936 1535 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:56:07.125967 1538 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:56:07.126298 1529 server_base.cc:1034] running on GCE node
I20260811 02:56:07.127022 1529 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 02:56:07.128216 1529 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 02:56:07.129426 1529 hybrid_clock.cc:648] HybridClock initialized: now 1786416967129391 us; error 47 us; skew 500 ppm
Aug 11 02:56:07 dist-test-slave-9gcw krb5kdc[1514](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416967, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.138.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 02:56:07.132539 1529 init.cc:377] Logged in from keytab as kudu/127.12.138.254@KRBTEST.COM (short username kudu)
I20260811 02:56:07.133783 1529 webserver.cc:460] Webserver started at http://127.12.138.254:42817/ using document root <none> and password file <none>
I20260811 02:56:07.134377 1529 fs_manager.cc:359] Metadata directory not provided
I20260811 02:56:07.134459 1529 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 02:56:07.134657 1529 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 02:56:07.136899 1529 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786416707705494-12843-0/minicluster-data/master-0/data/instance:
uuid: "95a1774bfadb4fc0967091217105576d"
format_stamp: "Formatted at 2026-08-11 02:56:07 on dist-test-slave-9gcw"
server_key: "21420b4b9ca4fa437de5f1333ab60587"
server_key_iv: "533f3960ded30206a5ab7c9c72ef0255"
server_key_version: "encryptionkey@0"
I20260811 02:56:07.137531 1529 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786416707705494-12843-0/minicluster-data/master-0/wal/instance:
uuid: "95a1774bfadb4fc0967091217105576d"
format_stamp: "Formatted at 2026-08-11 02:56:07 on dist-test-slave-9gcw"
server_key: "21420b4b9ca4fa437de5f1333ab60587"
server_key_iv: "533f3960ded30206a5ab7c9c72ef0255"
server_key_version: "encryptionkey@0"
I20260811 02:56:07.141502 1529 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260811 02:56:07.144361 1545 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 02:56:07.145756 1529 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 02:56:07.145861 1529 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786416707705494-12843-0/minicluster-data/master-0/data,/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786416707705494-12843-0/minicluster-data/master-0/wal
uuid: "95a1774bfadb4fc0967091217105576d"
format_stamp: "Formatted at 2026-08-11 02:56:07 on dist-test-slave-9gcw"
server_key: "21420b4b9ca4fa437de5f1333ab60587"
server_key_iv: "533f3960ded30206a5ab7c9c72ef0255"
server_key_version: "encryptionkey@0"
I20260811 02:56:07.145955 1529 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786416707705494-12843-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786416707705494-12843-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786416707705494-12843-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 02:56:07.161513 1529 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 02:56:07.164839 1529 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 02:56:07.164999 1529 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 02:56:07.172358 1529 rpc_server.cc:307] RPC server started. Bound to: 127.12.138.254:43779
I20260811 02:56:07.172386 1603 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.138.254:43779 every 8 connection(s)
I20260811 02:56:07.173434 1529 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786416707705494-12843-0/minicluster-data/master-0/data/info.pb
I20260811 02:56:07.173703 12843 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDawEwS/build/debug/bin/kudu as pid 1529
I20260811 02:56:07.173851 12843 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786416707705494-12843-0/minicluster-data/master-0/wal/instance
I20260811 02:56:07.174238 12843 external_mini_cluster.cc:1428] Setting key 0b682161b68ed06957cfdb19109c2fad
I20260811 02:56:07.176829 1604 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 02:56:07.182644 1604 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 95a1774bfadb4fc0967091217105576d: Bootstrap starting.
Aug 11 02:56:07 dist-test-slave-9gcw krb5kdc[1514](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416966, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.12.138.254@KRBTEST.COM
I20260811 02:56:07.185128 1604 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 95a1774bfadb4fc0967091217105576d: Neither blocks nor log segments found. Creating new log.
I20260811 02:56:07.186020 1604 log.cc:824] T 00000000000000000000000000000000 P 95a1774bfadb4fc0967091217105576d: Log is configured to *not* fsync() on all Append() calls
I20260811 02:56:07.188715 1604 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 95a1774bfadb4fc0967091217105576d: No bootstrap required, opened a new log
I20260811 02:56:07.189419 1607 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:56:07.175703 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33582 (local address 127.12.138.254:43779)
0811 02:56:07.176184 (+ 481us) server_negotiation.cc:206] Beginning negotiation
0811 02:56:07.176200 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0811 02:56:07.176235 (+ 35us) server_negotiation.cc:407] Connection header received
0811 02:56:07.177010 (+ 775us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:56:07.177030 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:56:07.177444 (+ 414us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:56:07.177789 (+ 345us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:56:07.178629 (+ 840us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:56:07.179671 (+ 1042us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:56:07.180550 (+ 879us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:56:07.181060 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:56:07.181291 (+ 231us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:56:07.184272 (+ 2981us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:56:07.184276 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:56:07.184288 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:56:07.184316 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:56:07.186341 (+ 2025us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:56:07.186851 (+ 510us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:56:07.186859 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:56:07.186866 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:56:07.186986 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:56:07.187289 (+ 303us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:56:07.187293 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:56:07.187294 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:56:07.187714 (+ 420us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:56:07.187924 (+ 210us) server_negotiation.cc:1035] Waiting for connection context
0811 02:56:07.188239 (+ 315us) server_negotiation.cc:299] Negotiation successful
0811 02:56:07.188540 (+ 301us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":327,"thread_start_us":152,"threads_started":1}
I20260811 02:56:07.193491 1604 raft_consensus.cc:348] T 00000000000000000000000000000000 P 95a1774bfadb4fc0967091217105576d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "95a1774bfadb4fc0967091217105576d" member_type: VOTER last_known_addr { host: "127.12.138.254" port: 43779 } }
I20260811 02:56:07.193759 1604 raft_consensus.cc:374] T 00000000000000000000000000000000 P 95a1774bfadb4fc0967091217105576d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 02:56:07.193853 1604 raft_consensus.cc:729] T 00000000000000000000000000000000 P 95a1774bfadb4fc0967091217105576d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 95a1774bfadb4fc0967091217105576d, State: Initialized, Role: FOLLOWER
I20260811 02:56:07.194371 1604 consensus_queue.cc:260] T 00000000000000000000000000000000 P 95a1774bfadb4fc0967091217105576d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "95a1774bfadb4fc0967091217105576d" member_type: VOTER last_known_addr { host: "127.12.138.254" port: 43779 } }
I20260811 02:56:07.194578 1604 raft_consensus.cc:388] T 00000000000000000000000000000000 P 95a1774bfadb4fc0967091217105576d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 02:56:07.194710 1604 raft_consensus.cc:482] T 00000000000000000000000000000000 P 95a1774bfadb4fc0967091217105576d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 02:56:07.194855 1604 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 95a1774bfadb4fc0967091217105576d [term 0 FOLLOWER]: Advancing to term 1
I20260811 02:56:07.196518 1604 raft_consensus.cc:504] T 00000000000000000000000000000000 P 95a1774bfadb4fc0967091217105576d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "95a1774bfadb4fc0967091217105576d" member_type: VOTER last_known_addr { host: "127.12.138.254" port: 43779 } }
I20260811 02:56:07.196877 1604 leader_election.cc:302] T 00000000000000000000000000000000 P 95a1774bfadb4fc0967091217105576d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 95a1774bfadb4fc0967091217105576d; no voters:
I20260811 02:56:07.197252 1604 leader_election.cc:288] T 00000000000000000000000000000000 P 95a1774bfadb4fc0967091217105576d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 02:56:07.197341 1609 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 95a1774bfadb4fc0967091217105576d [term 1 FOLLOWER]: Leader election won for term 1
I20260811 02:56:07.197640 1609 raft_consensus.cc:686] T 00000000000000000000000000000000 P 95a1774bfadb4fc0967091217105576d [term 1 LEADER]: Becoming Leader. State: Replica: 95a1774bfadb4fc0967091217105576d, State: Running, Role: LEADER
I20260811 02:56:07.197968 1609 consensus_queue.cc:237] T 00000000000000000000000000000000 P 95a1774bfadb4fc0967091217105576d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "95a1774bfadb4fc0967091217105576d" member_type: VOTER last_known_addr { host: "127.12.138.254" port: 43779 } }
I20260811 02:56:07.198742 1604 sys_catalog.cc:564] T 00000000000000000000000000000000 P 95a1774bfadb4fc0967091217105576d [sys.catalog]: configured and running, proceeding with master startup.
I20260811 02:56:07.199828 1610 sys_catalog.cc:455] T 00000000000000000000000000000000 P 95a1774bfadb4fc0967091217105576d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "95a1774bfadb4fc0967091217105576d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "95a1774bfadb4fc0967091217105576d" member_type: VOTER last_known_addr { host: "127.12.138.254" port: 43779 } } }
I20260811 02:56:07.199827 1611 sys_catalog.cc:455] T 00000000000000000000000000000000 P 95a1774bfadb4fc0967091217105576d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 95a1774bfadb4fc0967091217105576d. Latest consensus state: current_term: 1 leader_uuid: "95a1774bfadb4fc0967091217105576d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "95a1774bfadb4fc0967091217105576d" member_type: VOTER last_known_addr { host: "127.12.138.254" port: 43779 } } }
I20260811 02:56:07.200050 1610 sys_catalog.cc:458] T 00000000000000000000000000000000 P 95a1774bfadb4fc0967091217105576d [sys.catalog]: This master's current role is: LEADER
I20260811 02:56:07.200070 1611 sys_catalog.cc:458] T 00000000000000000000000000000000 P 95a1774bfadb4fc0967091217105576d [sys.catalog]: This master's current role is: LEADER
I20260811 02:56:07.201265 1619 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 02:56:07.204008 1619 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 02:56:07.209378 1619 catalog_manager.cc:1347] Generated new cluster ID: 9148a9fe47184152939f4f443a30d680
I20260811 02:56:07.209452 1619 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 02:56:07.222674 1619 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 02:56:07.224092 1619 catalog_manager.cc:1504] Loading token signing keys...
I20260811 02:56:07.233112 1619 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 95a1774bfadb4fc0967091217105576d: Generated new TSK 0
I20260811 02:56:07.234144 1619 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.12.138.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.138.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.138.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786416707705494-12843-0/krb5kdc/kudu_127.12.138.193.keytab.
Entry for principal kudu/127.12.138.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786416707705494-12843-0/krb5kdc/kudu_127.12.138.193.keytab.
I20260811 02:56:07.277866 12843 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDawEwS/build/debug/bin/kudu
/tmp/dist-test-taskDawEwS/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786416707705494-12843-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786416707705494-12843-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786416707705494-12843-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786416707705494-12843-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.138.193:0
--local_ip_for_outbound_sockets=127.12.138.193
--webserver_interface=127.12.138.193
--webserver_port=0
--tserver_master_addrs=127.12.138.254:43779
--keytab_file=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786416707705494-12843-0/krb5kdc/kudu_127.12.138.193.keytab
--principal=kudu/127.12.138.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.138.212:45985
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786416707705494-12843-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786416707705494-12843-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786416707705494-12843-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 02:56:07.396481 1633 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 02:56:07.396776 1633 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 02:56:07.396822 1633 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 02:56:07.401182 1633 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 02:56:07.401247 1633 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 02:56:07.401306 1633 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.138.193
I20260811 02:56:07.406713 1633 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.138.212:45985
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786416707705494-12843-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786416707705494-12843-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786416707705494-12843-0/krb5kdc/kudu_127.12.138.193.keytab
--principal=kudu/127.12.138.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.138.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786416707705494-12843-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.12.138.193
--webserver_port=0
--tserver_master_addrs=127.12.138.254:43779
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1633
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.138.193
--log_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786416707705494-12843-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 02:09:49 UTC on ccycloud.quasar-fsnqqc.root.comops.site
build id 4579570
I20260811 02:56:07.407724 1633 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 02:56:07.408689 1633 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 02:56:07.418419 1641 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:56:07.418500 1640 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:56:07.418504 1643 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:56:07.418781 1633 server_base.cc:1034] running on GCE node
I20260811 02:56:07.419232 1633 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 02:56:07.419948 1633 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 02:56:07.421195 1633 hybrid_clock.cc:648] HybridClock initialized: now 1786416967421140 us; error 34 us; skew 500 ppm
Aug 11 02:56:07 dist-test-slave-9gcw krb5kdc[1514](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416967, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.138.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 02:56:07.424296 1633 init.cc:377] Logged in from keytab as kudu/127.12.138.193@KRBTEST.COM (short username kudu)
I20260811 02:56:07.425417 1633 webserver.cc:460] Webserver started at http://127.12.138.193:39941/ using document root <none> and password file <none>
I20260811 02:56:07.425990 1633 fs_manager.cc:359] Metadata directory not provided
I20260811 02:56:07.426086 1633 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 02:56:07.426337 1633 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 02:56:07.428544 1633 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786416707705494-12843-0/minicluster-data/ts-0/data/instance:
uuid: "74701c073eb049f0b1dd6ee8ba2a9b58"
format_stamp: "Formatted at 2026-08-11 02:56:07 on dist-test-slave-9gcw"
server_key: "7dcc3d110899bbb9b09d9dd100efb730"
server_key_iv: "3389e8c73d28626fb96690b0a20a535c"
server_key_version: "encryptionkey@0"
I20260811 02:56:07.429133 1633 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786416707705494-12843-0/minicluster-data/ts-0/wal/instance:
uuid: "74701c073eb049f0b1dd6ee8ba2a9b58"
format_stamp: "Formatted at 2026-08-11 02:56:07 on dist-test-slave-9gcw"
server_key: "7dcc3d110899bbb9b09d9dd100efb730"
server_key_iv: "3389e8c73d28626fb96690b0a20a535c"
server_key_version: "encryptionkey@0"
I20260811 02:56:07.433055 1633 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.000s sys 0.003s
I20260811 02:56:07.435732 1650 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 02:56:07.437000 1633 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260811 02:56:07.437181 1633 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786416707705494-12843-0/minicluster-data/ts-0/data,/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786416707705494-12843-0/minicluster-data/ts-0/wal
uuid: "74701c073eb049f0b1dd6ee8ba2a9b58"
format_stamp: "Formatted at 2026-08-11 02:56:07 on dist-test-slave-9gcw"
server_key: "7dcc3d110899bbb9b09d9dd100efb730"
server_key_iv: "3389e8c73d28626fb96690b0a20a535c"
server_key_version: "encryptionkey@0"
I20260811 02:56:07.437341 1633 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786416707705494-12843-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786416707705494-12843-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786416707705494-12843-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 02:56:07.453848 1633 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 02:56:07.457473 1633 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 02:56:07.457711 1633 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 02:56:07.458384 1633 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 02:56:07.459362 1633 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 02:56:07.459443 1633 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 02:56:07.459525 1633 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 02:56:07.459582 1633 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 02:56:07.470073 1633 rpc_server.cc:307] RPC server started. Bound to: 127.12.138.193:44263
I20260811 02:56:07.470099 1775 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.138.193:44263 every 8 connection(s)
I20260811 02:56:07.471163 1633 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786416707705494-12843-0/minicluster-data/ts-0/data/info.pb
I20260811 02:56:07.476051 12843 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDawEwS/build/debug/bin/kudu as pid 1633
I20260811 02:56:07.476218 12843 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786416707705494-12843-0/minicluster-data/ts-0/wal/instance
I20260811 02:56:07.476521 12843 external_mini_cluster.cc:1428] Setting key 57e6173b22b391939ab7b7fb2ac59d1a
Aug 11 02:56:07 dist-test-slave-9gcw krb5kdc[1514](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416967, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.138.193@KRBTEST.COM for kudu/127.12.138.254@KRBTEST.COM
I20260811 02:56:07.487818 1607 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:56:07.473248 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.138.193:35163 (local address 127.12.138.254:43779)
0811 02:56:07.473395 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0811 02:56:07.473399 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 02:56:07.474253 (+ 854us) server_negotiation.cc:407] Connection header received
0811 02:56:07.475611 (+ 1358us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:56:07.475614 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:56:07.475664 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:56:07.475765 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:56:07.477233 (+ 1468us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:56:07.477867 (+ 634us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:56:07.479310 (+ 1443us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:56:07.479715 (+ 405us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:56:07.479841 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:56:07.483567 (+ 3726us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:56:07.483575 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:56:07.483581 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:56:07.483618 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:56:07.485471 (+ 1853us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:56:07.486013 (+ 542us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:56:07.486018 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:56:07.486022 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:56:07.486121 (+ 99us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:56:07.486501 (+ 380us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:56:07.486507 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:56:07.486510 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:56:07.486785 (+ 275us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:56:07.486894 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0811 02:56:07.487431 (+ 537us) server_negotiation.cc:299] Negotiation successful
0811 02:56:07.487585 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
WARNING: no policy specified for kudu/127.12.138.194@KRBTEST.COM; defaulting to no policy
I20260811 02:56:07.488521 1778 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:56:07.473613 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.138.254:43779 (local address 127.12.138.193:35163)
0811 02:56:07.474076 (+ 463us) negotiation.cc:107] Waiting for socket to connect
0811 02:56:07.474120 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0811 02:56:07.475402 (+ 1282us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:56:07.475929 (+ 527us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:56:07.475936 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:56:07.476436 (+ 500us) client_negotiation.cc:189] Negotiated authn=SASL
0811 02:56:07.477042 (+ 606us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:56:07.477066 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:56:07.478011 (+ 945us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:56:07.478014 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:56:07.479158 (+ 1144us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:56:07.479169 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:56:07.479912 (+ 743us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:56:07.479918 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:56:07.480160 (+ 242us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:56:07.481499 (+ 1339us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 02:56:07.481521 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0811 02:56:07.483368 (+ 1847us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 02:56:07.485599 (+ 2231us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:56:07.485604 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:56:07.485617 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:56:07.485890 (+ 273us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:56:07.486221 (+ 331us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:56:07.486223 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:56:07.486225 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:56:07.486385 (+ 160us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:56:07.486891 (+ 506us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 02:56:07.486896 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 02:56:07.487143 (+ 247us) client_negotiation.cc:769] Sending connection context
0811 02:56:07.487334 (+ 191us) client_negotiation.cc:240] Negotiation successful
0811 02:56:07.487604 (+ 270us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":271,"thread_start_us":117,"threads_started":1}
I20260811 02:56:07.490041 1776 heartbeater.cc:347] Connected to a master server at 127.12.138.254:43779
I20260811 02:56:07.490329 1776 heartbeater.cc:464] Registering TS with master...
I20260811 02:56:07.490888 1776 heartbeater.cc:511] Master 127.12.138.254:43779 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.138.194@KRBTEST.COM" created.
I20260811 02:56:07.492753 1563 ts_manager.cc:194] Registered new tserver with Master: 74701c073eb049f0b1dd6ee8ba2a9b58 (127.12.138.193:44263)
I20260811 02:56:07.494246 1563 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.12.138.193@KRBTEST.COM'} at 127.12.138.193:35163
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.138.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786416707705494-12843-0/krb5kdc/kudu_127.12.138.194.keytab.
Entry for principal kudu/127.12.138.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786416707705494-12843-0/krb5kdc/kudu_127.12.138.194.keytab.
I20260811 02:56:07.509375 12843 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDawEwS/build/debug/bin/kudu
/tmp/dist-test-taskDawEwS/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786416707705494-12843-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786416707705494-12843-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786416707705494-12843-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786416707705494-12843-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.138.194:0
--local_ip_for_outbound_sockets=127.12.138.194
--webserver_interface=127.12.138.194
--webserver_port=0
--tserver_master_addrs=127.12.138.254:43779
--keytab_file=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786416707705494-12843-0/krb5kdc/kudu_127.12.138.194.keytab
--principal=kudu/127.12.138.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.138.212:45985
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786416707705494-12843-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786416707705494-12843-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786416707705494-12843-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 02:56:07.626545 1781 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 02:56:07.626878 1781 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 02:56:07.626971 1781 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 02:56:07.631157 1781 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 02:56:07.631251 1781 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 02:56:07.631340 1781 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.138.194
I20260811 02:56:07.636790 1781 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.138.212:45985
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786416707705494-12843-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786416707705494-12843-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786416707705494-12843-0/krb5kdc/kudu_127.12.138.194.keytab
--principal=kudu/127.12.138.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.138.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786416707705494-12843-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.12.138.194
--webserver_port=0
--tserver_master_addrs=127.12.138.254:43779
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1781
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.138.194
--log_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786416707705494-12843-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 02:09:49 UTC on ccycloud.quasar-fsnqqc.root.comops.site
build id 4579570
I20260811 02:56:07.638054 1781 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 02:56:07.639117 1781 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 02:56:07.649098 1788 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:56:07.649106 1789 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:56:07.649106 1791 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:56:07.649447 1781 server_base.cc:1034] running on GCE node
I20260811 02:56:07.649921 1781 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 02:56:07.650570 1781 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 02:56:07.651742 1781 hybrid_clock.cc:648] HybridClock initialized: now 1786416967651711 us; error 43 us; skew 500 ppm
Aug 11 02:56:07 dist-test-slave-9gcw krb5kdc[1514](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416967, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.138.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 02:56:07.654779 1781 init.cc:377] Logged in from keytab as kudu/127.12.138.194@KRBTEST.COM (short username kudu)
I20260811 02:56:07.655660 1781 webserver.cc:460] Webserver started at http://127.12.138.194:37353/ using document root <none> and password file <none>
I20260811 02:56:07.656281 1781 fs_manager.cc:359] Metadata directory not provided
I20260811 02:56:07.656347 1781 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 02:56:07.656520 1781 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 02:56:07.658737 1781 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786416707705494-12843-0/minicluster-data/ts-1/data/instance:
uuid: "0563bdf80aa64bff8b29424496bc9d76"
format_stamp: "Formatted at 2026-08-11 02:56:07 on dist-test-slave-9gcw"
server_key: "b9fd4c8ba0a30a358493fad55685e32c"
server_key_iv: "696fb016cdc74261f25b769d56512040"
server_key_version: "encryptionkey@0"
I20260811 02:56:07.659268 1781 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786416707705494-12843-0/minicluster-data/ts-1/wal/instance:
uuid: "0563bdf80aa64bff8b29424496bc9d76"
format_stamp: "Formatted at 2026-08-11 02:56:07 on dist-test-slave-9gcw"
server_key: "b9fd4c8ba0a30a358493fad55685e32c"
server_key_iv: "696fb016cdc74261f25b769d56512040"
server_key_version: "encryptionkey@0"
I20260811 02:56:07.663090 1781 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260811 02:56:07.665643 1799 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 02:56:07.667047 1781 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 02:56:07.667179 1781 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786416707705494-12843-0/minicluster-data/ts-1/data,/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786416707705494-12843-0/minicluster-data/ts-1/wal
uuid: "0563bdf80aa64bff8b29424496bc9d76"
format_stamp: "Formatted at 2026-08-11 02:56:07 on dist-test-slave-9gcw"
server_key: "b9fd4c8ba0a30a358493fad55685e32c"
server_key_iv: "696fb016cdc74261f25b769d56512040"
server_key_version: "encryptionkey@0"
I20260811 02:56:07.667299 1781 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786416707705494-12843-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786416707705494-12843-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786416707705494-12843-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 02:56:07.676326 1781 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 02:56:07.679270 1781 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 02:56:07.679420 1781 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 02:56:07.679929 1781 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 02:56:07.680691 1781 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 02:56:07.680747 1781 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 02:56:07.680786 1781 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 02:56:07.680809 1781 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 02:56:07.689872 1781 rpc_server.cc:307] RPC server started. Bound to: 127.12.138.194:45285
I20260811 02:56:07.689956 1930 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.138.194:45285 every 8 connection(s)
I20260811 02:56:07.690778 1781 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786416707705494-12843-0/minicluster-data/ts-1/data/info.pb
I20260811 02:56:07.696977 12843 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDawEwS/build/debug/bin/kudu as pid 1781
I20260811 02:56:07.697106 12843 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786416707705494-12843-0/minicluster-data/ts-1/wal/instance
I20260811 02:56:07.697471 12843 external_mini_cluster.cc:1428] Setting key 93d766a18a89201faeb9d0ff7cafc906
Aug 11 02:56:07 dist-test-slave-9gcw krb5kdc[1514](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416967, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.138.194@KRBTEST.COM for kudu/127.12.138.254@KRBTEST.COM
I20260811 02:56:07.705937 1607 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:56:07.692640 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.138.194:37161 (local address 127.12.138.254:43779)
0811 02:56:07.692803 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0811 02:56:07.692808 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 02:56:07.693702 (+ 894us) server_negotiation.cc:407] Connection header received
0811 02:56:07.695154 (+ 1452us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:56:07.695157 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:56:07.695225 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:56:07.695323 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:56:07.696513 (+ 1190us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:56:07.697090 (+ 577us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:56:07.698588 (+ 1498us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:56:07.698891 (+ 303us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:56:07.699002 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:56:07.701965 (+ 2963us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:56:07.701969 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:56:07.701972 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:56:07.701999 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:56:07.703496 (+ 1497us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:56:07.704161 (+ 665us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:56:07.704164 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:56:07.704165 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:56:07.704244 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:56:07.704659 (+ 415us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:56:07.704661 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:56:07.704662 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:56:07.704843 (+ 181us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:56:07.704922 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0811 02:56:07.705679 (+ 757us) server_negotiation.cc:299] Negotiation successful
0811 02:56:07.705800 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260811 02:56:07.707176 1933 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:56:07.693061 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.138.254:43779 (local address 127.12.138.194:37161)
0811 02:56:07.693553 (+ 492us) negotiation.cc:107] Waiting for socket to connect
0811 02:56:07.693595 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0811 02:56:07.694929 (+ 1334us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:56:07.695510 (+ 581us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:56:07.695518 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:56:07.695879 (+ 361us) client_negotiation.cc:189] Negotiated authn=SASL
0811 02:56:07.696354 (+ 475us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:56:07.696367 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:56:07.697271 (+ 904us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:56:07.697276 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:56:07.698471 (+ 1195us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:56:07.698483 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:56:07.699041 (+ 558us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:56:07.699045 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:56:07.699255 (+ 210us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:56:07.699860 (+ 605us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 02:56:07.699877 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 02:56:07.701802 (+ 1925us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 02:56:07.703617 (+ 1815us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:56:07.703626 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:56:07.703644 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:56:07.704054 (+ 410us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:56:07.704339 (+ 285us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:56:07.704345 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:56:07.704350 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:56:07.704553 (+ 203us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:56:07.704953 (+ 400us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 02:56:07.704959 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 02:56:07.705337 (+ 378us) client_negotiation.cc:769] Sending connection context
0811 02:56:07.705601 (+ 264us) client_negotiation.cc:240] Negotiation successful
0811 02:56:07.706002 (+ 401us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":304,"thread_start_us":110,"threads_started":1}
I20260811 02:56:07.708483 1931 heartbeater.cc:347] Connected to a master server at 127.12.138.254:43779
I20260811 02:56:07.708807 1931 heartbeater.cc:464] Registering TS with master...
WARNING: no policy specified for kudu/127.12.138.195@KRBTEST.COM; defaulting to no policy
I20260811 02:56:07.709429 1931 heartbeater.cc:511] Master 127.12.138.254:43779 requested a full tablet report, sending...
I20260811 02:56:07.710533 1563 ts_manager.cc:194] Registered new tserver with Master: 0563bdf80aa64bff8b29424496bc9d76 (127.12.138.194:45285)
I20260811 02:56:07.711153 1563 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.12.138.194@KRBTEST.COM'} at 127.12.138.194:37161
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.138.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.138.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786416707705494-12843-0/krb5kdc/kudu_127.12.138.195.keytab.
Entry for principal kudu/127.12.138.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786416707705494-12843-0/krb5kdc/kudu_127.12.138.195.keytab.
I20260811 02:56:07.728641 12843 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDawEwS/build/debug/bin/kudu
/tmp/dist-test-taskDawEwS/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786416707705494-12843-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786416707705494-12843-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786416707705494-12843-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786416707705494-12843-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.138.195:0
--local_ip_for_outbound_sockets=127.12.138.195
--webserver_interface=127.12.138.195
--webserver_port=0
--tserver_master_addrs=127.12.138.254:43779
--keytab_file=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786416707705494-12843-0/krb5kdc/kudu_127.12.138.195.keytab
--principal=kudu/127.12.138.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.138.212:45985
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786416707705494-12843-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786416707705494-12843-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786416707705494-12843-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 02:56:07.845472 1936 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 02:56:07.845737 1936 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 02:56:07.845815 1936 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 02:56:07.849939 1936 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 02:56:07.850010 1936 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 02:56:07.850095 1936 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.138.195
I20260811 02:56:07.855865 1936 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.138.212:45985
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786416707705494-12843-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786416707705494-12843-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786416707705494-12843-0/krb5kdc/kudu_127.12.138.195.keytab
--principal=kudu/127.12.138.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.138.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786416707705494-12843-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.12.138.195
--webserver_port=0
--tserver_master_addrs=127.12.138.254:43779
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1936
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.138.195
--log_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786416707705494-12843-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 02:09:49 UTC on ccycloud.quasar-fsnqqc.root.comops.site
build id 4579570
I20260811 02:56:07.857007 1936 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 02:56:07.858245 1936 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 02:56:07.868444 1941 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:56:07.868431 1942 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:56:07.868525 1945 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:56:07.868738 1936 server_base.cc:1034] running on GCE node
I20260811 02:56:07.869218 1936 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 02:56:07.869890 1936 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 02:56:07.871101 1936 hybrid_clock.cc:648] HybridClock initialized: now 1786416967871087 us; error 46 us; skew 500 ppm
Aug 11 02:56:07 dist-test-slave-9gcw krb5kdc[1514](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416967, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.138.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 02:56:07.874109 1936 init.cc:377] Logged in from keytab as kudu/127.12.138.195@KRBTEST.COM (short username kudu)
I20260811 02:56:07.875022 1936 webserver.cc:460] Webserver started at http://127.12.138.195:40749/ using document root <none> and password file <none>
I20260811 02:56:07.875633 1936 fs_manager.cc:359] Metadata directory not provided
I20260811 02:56:07.875707 1936 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 02:56:07.875891 1936 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 02:56:07.878003 1936 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786416707705494-12843-0/minicluster-data/ts-2/data/instance:
uuid: "5df3a0a85d0c49f6b582a258e98d025b"
format_stamp: "Formatted at 2026-08-11 02:56:07 on dist-test-slave-9gcw"
server_key: "d829ed30d8f529d213c39b95423cd2fc"
server_key_iv: "ab61798881aa1dfb8f95260af9e80d82"
server_key_version: "encryptionkey@0"
I20260811 02:56:07.878583 1936 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786416707705494-12843-0/minicluster-data/ts-2/wal/instance:
uuid: "5df3a0a85d0c49f6b582a258e98d025b"
format_stamp: "Formatted at 2026-08-11 02:56:07 on dist-test-slave-9gcw"
server_key: "d829ed30d8f529d213c39b95423cd2fc"
server_key_iv: "ab61798881aa1dfb8f95260af9e80d82"
server_key_version: "encryptionkey@0"
I20260811 02:56:07.882575 1936 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260811 02:56:07.885191 1953 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 02:56:07.886687 1936 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 02:56:07.886842 1936 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786416707705494-12843-0/minicluster-data/ts-2/data,/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786416707705494-12843-0/minicluster-data/ts-2/wal
uuid: "5df3a0a85d0c49f6b582a258e98d025b"
format_stamp: "Formatted at 2026-08-11 02:56:07 on dist-test-slave-9gcw"
server_key: "d829ed30d8f529d213c39b95423cd2fc"
server_key_iv: "ab61798881aa1dfb8f95260af9e80d82"
server_key_version: "encryptionkey@0"
I20260811 02:56:07.886977 1936 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786416707705494-12843-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786416707705494-12843-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786416707705494-12843-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 02:56:07.901695 1936 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 02:56:07.905606 1936 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 02:56:07.905857 1936 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 02:56:07.906685 1936 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 02:56:07.907996 1936 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 02:56:07.908053 1936 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 02:56:07.908123 1936 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 02:56:07.908167 1936 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 02:56:07.918529 1936 rpc_server.cc:307] RPC server started. Bound to: 127.12.138.195:39337
I20260811 02:56:07.918550 2081 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.138.195:39337 every 8 connection(s)
I20260811 02:56:07.919544 1936 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786416707705494-12843-0/minicluster-data/ts-2/data/info.pb
I20260811 02:56:07.926633 12843 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDawEwS/build/debug/bin/kudu as pid 1936
I20260811 02:56:07.926798 12843 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786416707705494-12843-0/minicluster-data/ts-2/wal/instance
I20260811 02:56:07.927093 12843 external_mini_cluster.cc:1428] Setting key f203c71af2df03f839e9b1bf6816f8d6
Aug 11 02:56:07 dist-test-slave-9gcw krb5kdc[1514](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416967, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.138.195@KRBTEST.COM for kudu/127.12.138.254@KRBTEST.COM
I20260811 02:56:07.935971 1607 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:56:07.921919 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.138.195:55147 (local address 127.12.138.254:43779)
0811 02:56:07.922106 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0811 02:56:07.922110 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 02:56:07.923217 (+ 1107us) server_negotiation.cc:407] Connection header received
0811 02:56:07.924892 (+ 1675us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:56:07.924896 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:56:07.924968 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:56:07.925108 (+ 140us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:56:07.926558 (+ 1450us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:56:07.927358 (+ 800us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:56:07.929085 (+ 1727us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:56:07.929442 (+ 357us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:56:07.929583 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:56:07.932382 (+ 2799us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:56:07.932391 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:56:07.932393 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:56:07.932424 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:56:07.933848 (+ 1424us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:56:07.934334 (+ 486us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:56:07.934337 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:56:07.934338 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:56:07.934422 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:56:07.934761 (+ 339us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:56:07.934764 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:56:07.934765 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:56:07.934962 (+ 197us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:56:07.935084 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0811 02:56:07.935702 (+ 618us) server_negotiation.cc:299] Negotiation successful
0811 02:56:07.935833 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260811 02:56:07.936764 2084 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:56:07.922349 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.138.254:43779 (local address 127.12.138.195:55147)
0811 02:56:07.923016 (+ 667us) negotiation.cc:107] Waiting for socket to connect
0811 02:56:07.923068 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0811 02:56:07.924622 (+ 1554us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:56:07.925350 (+ 728us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:56:07.925366 (+ 16us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:56:07.925787 (+ 421us) client_negotiation.cc:189] Negotiated authn=SASL
0811 02:56:07.926380 (+ 593us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:56:07.926398 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:56:07.927503 (+ 1105us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:56:07.927509 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:56:07.928919 (+ 1410us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:56:07.928930 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:56:07.929641 (+ 711us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:56:07.929647 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:56:07.929857 (+ 210us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:56:07.930485 (+ 628us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 02:56:07.930504 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0811 02:56:07.932219 (+ 1715us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 02:56:07.933957 (+ 1738us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:56:07.933962 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:56:07.933974 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:56:07.934238 (+ 264us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:56:07.934531 (+ 293us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:56:07.934534 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:56:07.934536 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:56:07.934680 (+ 144us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:56:07.935114 (+ 434us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 02:56:07.935121 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 02:56:07.935426 (+ 305us) client_negotiation.cc:769] Sending connection context
0811 02:56:07.935592 (+ 166us) client_negotiation.cc:240] Negotiation successful
0811 02:56:07.935889 (+ 297us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":370,"thread_start_us":108,"threads_started":1}
I20260811 02:56:07.938092 2082 heartbeater.cc:347] Connected to a master server at 127.12.138.254:43779
I20260811 02:56:07.938354 2082 heartbeater.cc:464] Registering TS with master...
I20260811 02:56:07.938946 2082 heartbeater.cc:511] Master 127.12.138.254:43779 requested a full tablet report, sending...
I20260811 02:56:07.940165 1563 ts_manager.cc:194] Registered new tserver with Master: 5df3a0a85d0c49f6b582a258e98d025b (127.12.138.195:39337)
I20260811 02:56:07.940907 1563 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.12.138.195@KRBTEST.COM'} at 127.12.138.195:55147
I20260811 02:56:07.941421 12843 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 02:56:07.951538 1607 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:56:07.943713 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33594 (local address 127.12.138.254:43779)
0811 02:56:07.943835 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0811 02:56:07.943840 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 02:56:07.944020 (+ 180us) server_negotiation.cc:407] Connection header received
0811 02:56:07.944172 (+ 152us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:56:07.944177 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:56:07.944218 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:56:07.944308 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:56:07.945011 (+ 703us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:56:07.945579 (+ 568us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:56:07.946439 (+ 860us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:56:07.946677 (+ 238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:56:07.946784 (+ 107us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:56:07.948385 (+ 1601us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:56:07.948389 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:56:07.948391 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:56:07.948416 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:56:07.949792 (+ 1376us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:56:07.950291 (+ 499us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:56:07.950293 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:56:07.950295 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:56:07.950370 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:56:07.950748 (+ 378us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:56:07.950751 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:56:07.950752 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:56:07.950957 (+ 205us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:56:07.951094 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0811 02:56:07.951332 (+ 238us) server_negotiation.cc:299] Negotiation successful
0811 02:56:07.951416 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260811 02:56:07.954830 1563 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33594:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 02:56:07.958039 1563 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 02:56:07.973594 2093 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:56:07.967247 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.138.194:45285 (local address 127.0.0.1:49360)
0811 02:56:07.967582 (+ 335us) negotiation.cc:107] Waiting for socket to connect
0811 02:56:07.967630 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0811 02:56:07.967775 (+ 145us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:56:07.969182 (+ 1407us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:56:07.969188 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:56:07.969215 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 02:56:07.969400 (+ 185us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:56:07.969410 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:56:07.970699 (+ 1289us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:56:07.970703 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:56:07.971994 (+ 1291us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:56:07.972004 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:56:07.972955 (+ 951us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:56:07.972958 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:56:07.973081 (+ 123us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:56:07.973125 (+ 44us) client_negotiation.cc:769] Sending connection context
0811 02:56:07.973293 (+ 168us) client_negotiation.cc:240] Negotiation successful
0811 02:56:07.973418 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":162,"mutex_wait_us":2,"spinlock_wait_cycles":2944,"thread_start_us":92,"threads_started":1}
I20260811 02:56:07.973603 2096 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:56:07.967966 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49360 (local address 127.12.138.194:45285)
0811 02:56:07.968368 (+ 402us) server_negotiation.cc:206] Beginning negotiation
0811 02:56:07.968381 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0811 02:56:07.968406 (+ 25us) server_negotiation.cc:407] Connection header received
0811 02:56:07.968491 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:56:07.968496 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:56:07.968752 (+ 256us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:56:07.968923 (+ 171us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 02:56:07.969539 (+ 616us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:56:07.970559 (+ 1020us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:56:07.972113 (+ 1554us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:56:07.972797 (+ 684us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:56:07.972928 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:56:07.973071 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0811 02:56:07.973330 (+ 259us) server_negotiation.cc:299] Negotiation successful
0811 02:56:07.973438 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":270,"thread_start_us":169,"threads_started":1}
I20260811 02:56:07.974427 2094 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:56:07.967253 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.138.195:39337 (local address 127.0.0.1:34696)
0811 02:56:07.968434 (+ 1181us) negotiation.cc:107] Waiting for socket to connect
0811 02:56:07.968448 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 02:56:07.968542 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:56:07.969222 (+ 680us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:56:07.969228 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:56:07.969253 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 02:56:07.969443 (+ 190us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:56:07.969454 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:56:07.971178 (+ 1724us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:56:07.971181 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:56:07.972640 (+ 1459us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:56:07.972648 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:56:07.973984 (+ 1336us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:56:07.973987 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:56:07.974062 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:56:07.974132 (+ 70us) client_negotiation.cc:769] Sending connection context
0811 02:56:07.974218 (+ 86us) client_negotiation.cc:240] Negotiation successful
0811 02:56:07.974292 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1115,"mutex_wait_us":95,"thread_start_us":119,"threads_started":1}
I20260811 02:56:07.974763 2095 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:56:07.967727 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34696 (local address 127.12.138.195:39337)
0811 02:56:07.968205 (+ 478us) server_negotiation.cc:206] Beginning negotiation
0811 02:56:07.968217 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 02:56:07.968651 (+ 434us) server_negotiation.cc:407] Connection header received
0811 02:56:07.968740 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:56:07.968745 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:56:07.968939 (+ 194us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:56:07.969104 (+ 165us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 02:56:07.969661 (+ 557us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:56:07.970980 (+ 1319us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:56:07.972770 (+ 1790us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:56:07.973673 (+ 903us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:56:07.973816 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:56:07.973928 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0811 02:56:07.974252 (+ 324us) server_negotiation.cc:299] Negotiation successful
0811 02:56:07.974645 (+ 393us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":359,"thread_start_us":152,"threads_started":1}
I20260811 02:56:07.974758 2098 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:56:07.968880 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45614 (local address 127.12.138.193:44263)
0811 02:56:07.969216 (+ 336us) server_negotiation.cc:206] Beginning negotiation
0811 02:56:07.969237 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0811 02:56:07.969387 (+ 150us) server_negotiation.cc:407] Connection header received
0811 02:56:07.969496 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:56:07.969503 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:56:07.969698 (+ 195us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:56:07.969871 (+ 173us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 02:56:07.970068 (+ 197us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:56:07.971219 (+ 1151us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:56:07.972791 (+ 1572us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:56:07.973743 (+ 952us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:56:07.973867 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:56:07.973983 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0811 02:56:07.974498 (+ 515us) server_negotiation.cc:299] Negotiation successful
0811 02:56:07.974603 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":202,"thread_start_us":87,"threads_started":1}
I20260811 02:56:07.975064 2097 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:56:07.968452 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.138.193:44263 (local address 127.0.0.1:45614)
0811 02:56:07.969270 (+ 818us) negotiation.cc:107] Waiting for socket to connect
0811 02:56:07.969284 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 02:56:07.969387 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:56:07.969858 (+ 471us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:56:07.969861 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:56:07.969880 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 02:56:07.969971 (+ 91us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:56:07.969977 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:56:07.971362 (+ 1385us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:56:07.971365 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:56:07.972628 (+ 1263us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:56:07.972650 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:56:07.974057 (+ 1407us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:56:07.974061 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:56:07.974187 (+ 126us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:56:07.974209 (+ 22us) client_negotiation.cc:769] Sending connection context
0811 02:56:07.974279 (+ 70us) client_negotiation.cc:240] Negotiation successful
0811 02:56:07.974361 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":754,"mutex_wait_us":43,"thread_start_us":95,"threads_started":1}
I20260811 02:56:07.977710 1703 tablet_service.cc:1467] Processing CreateTablet for tablet eec61a63b4bf409cbb278b21c26eb829 (DEFAULT_TABLE table=test-table [id=3586fb46aa7a4028a36840c88a0545d3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 02:56:07.977715 2009 tablet_service.cc:1467] Processing CreateTablet for tablet eec61a63b4bf409cbb278b21c26eb829 (DEFAULT_TABLE table=test-table [id=3586fb46aa7a4028a36840c88a0545d3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 02:56:07.977715 1855 tablet_service.cc:1467] Processing CreateTablet for tablet eec61a63b4bf409cbb278b21c26eb829 (DEFAULT_TABLE table=test-table [id=3586fb46aa7a4028a36840c88a0545d3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 02:56:07.978771 1703 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet eec61a63b4bf409cbb278b21c26eb829. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 02:56:07.978782 2009 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet eec61a63b4bf409cbb278b21c26eb829. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 02:56:07.978782 1855 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet eec61a63b4bf409cbb278b21c26eb829. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 02:56:07.985459 2099 tablet_bootstrap.cc:492] T eec61a63b4bf409cbb278b21c26eb829 P 74701c073eb049f0b1dd6ee8ba2a9b58: Bootstrap starting.
I20260811 02:56:07.986835 2101 tablet_bootstrap.cc:492] T eec61a63b4bf409cbb278b21c26eb829 P 5df3a0a85d0c49f6b582a258e98d025b: Bootstrap starting.
I20260811 02:56:07.987672 2099 tablet_bootstrap.cc:654] T eec61a63b4bf409cbb278b21c26eb829 P 74701c073eb049f0b1dd6ee8ba2a9b58: Neither blocks nor log segments found. Creating new log.
I20260811 02:56:07.988560 2101 tablet_bootstrap.cc:654] T eec61a63b4bf409cbb278b21c26eb829 P 5df3a0a85d0c49f6b582a258e98d025b: Neither blocks nor log segments found. Creating new log.
I20260811 02:56:07.988847 2099 log.cc:824] T eec61a63b4bf409cbb278b21c26eb829 P 74701c073eb049f0b1dd6ee8ba2a9b58: Log is configured to *not* fsync() on all Append() calls
I20260811 02:56:07.989530 2100 tablet_bootstrap.cc:492] T eec61a63b4bf409cbb278b21c26eb829 P 0563bdf80aa64bff8b29424496bc9d76: Bootstrap starting.
I20260811 02:56:07.989845 2101 log.cc:824] T eec61a63b4bf409cbb278b21c26eb829 P 5df3a0a85d0c49f6b582a258e98d025b: Log is configured to *not* fsync() on all Append() calls
I20260811 02:56:07.991065 2099 tablet_bootstrap.cc:492] T eec61a63b4bf409cbb278b21c26eb829 P 74701c073eb049f0b1dd6ee8ba2a9b58: No bootstrap required, opened a new log
I20260811 02:56:07.991299 2099 ts_tablet_manager.cc:1397] T eec61a63b4bf409cbb278b21c26eb829 P 74701c073eb049f0b1dd6ee8ba2a9b58: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260811 02:56:07.991967 2101 tablet_bootstrap.cc:492] T eec61a63b4bf409cbb278b21c26eb829 P 5df3a0a85d0c49f6b582a258e98d025b: No bootstrap required, opened a new log
I20260811 02:56:07.992046 2100 tablet_bootstrap.cc:654] T eec61a63b4bf409cbb278b21c26eb829 P 0563bdf80aa64bff8b29424496bc9d76: Neither blocks nor log segments found. Creating new log.
I20260811 02:56:07.992156 2101 ts_tablet_manager.cc:1397] T eec61a63b4bf409cbb278b21c26eb829 P 5df3a0a85d0c49f6b582a258e98d025b: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.004s
I20260811 02:56:07.993430 2100 log.cc:824] T eec61a63b4bf409cbb278b21c26eb829 P 0563bdf80aa64bff8b29424496bc9d76: Log is configured to *not* fsync() on all Append() calls
I20260811 02:56:07.994907 2099 raft_consensus.cc:348] T eec61a63b4bf409cbb278b21c26eb829 P 74701c073eb049f0b1dd6ee8ba2a9b58 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0563bdf80aa64bff8b29424496bc9d76" member_type: VOTER last_known_addr { host: "127.12.138.194" port: 45285 } } peers { permanent_uuid: "5df3a0a85d0c49f6b582a258e98d025b" member_type: VOTER last_known_addr { host: "127.12.138.195" port: 39337 } } peers { permanent_uuid: "74701c073eb049f0b1dd6ee8ba2a9b58" member_type: VOTER last_known_addr { host: "127.12.138.193" port: 44263 } }
I20260811 02:56:07.995150 2099 raft_consensus.cc:374] T eec61a63b4bf409cbb278b21c26eb829 P 74701c073eb049f0b1dd6ee8ba2a9b58 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 02:56:07.995204 2099 raft_consensus.cc:729] T eec61a63b4bf409cbb278b21c26eb829 P 74701c073eb049f0b1dd6ee8ba2a9b58 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 74701c073eb049f0b1dd6ee8ba2a9b58, State: Initialized, Role: FOLLOWER
I20260811 02:56:07.995539 2100 tablet_bootstrap.cc:492] T eec61a63b4bf409cbb278b21c26eb829 P 0563bdf80aa64bff8b29424496bc9d76: No bootstrap required, opened a new log
I20260811 02:56:07.995730 2100 ts_tablet_manager.cc:1397] T eec61a63b4bf409cbb278b21c26eb829 P 0563bdf80aa64bff8b29424496bc9d76: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260811 02:56:07.995721 2099 consensus_queue.cc:260] T eec61a63b4bf409cbb278b21c26eb829 P 74701c073eb049f0b1dd6ee8ba2a9b58 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0563bdf80aa64bff8b29424496bc9d76" member_type: VOTER last_known_addr { host: "127.12.138.194" port: 45285 } } peers { permanent_uuid: "5df3a0a85d0c49f6b582a258e98d025b" member_type: VOTER last_known_addr { host: "127.12.138.195" port: 39337 } } peers { permanent_uuid: "74701c073eb049f0b1dd6ee8ba2a9b58" member_type: VOTER last_known_addr { host: "127.12.138.193" port: 44263 } }
I20260811 02:56:07.996057 2101 raft_consensus.cc:348] T eec61a63b4bf409cbb278b21c26eb829 P 5df3a0a85d0c49f6b582a258e98d025b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0563bdf80aa64bff8b29424496bc9d76" member_type: VOTER last_known_addr { host: "127.12.138.194" port: 45285 } } peers { permanent_uuid: "5df3a0a85d0c49f6b582a258e98d025b" member_type: VOTER last_known_addr { host: "127.12.138.195" port: 39337 } } peers { permanent_uuid: "74701c073eb049f0b1dd6ee8ba2a9b58" member_type: VOTER last_known_addr { host: "127.12.138.193" port: 44263 } }
I20260811 02:56:07.996305 2101 raft_consensus.cc:374] T eec61a63b4bf409cbb278b21c26eb829 P 5df3a0a85d0c49f6b582a258e98d025b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 02:56:07.996363 2101 raft_consensus.cc:729] T eec61a63b4bf409cbb278b21c26eb829 P 5df3a0a85d0c49f6b582a258e98d025b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5df3a0a85d0c49f6b582a258e98d025b, State: Initialized, Role: FOLLOWER
I20260811 02:56:07.996380 1776 heartbeater.cc:503] Master 127.12.138.254:43779 was elected leader, sending a full tablet report...
I20260811 02:56:07.996742 2099 ts_tablet_manager.cc:1428] T eec61a63b4bf409cbb278b21c26eb829 P 74701c073eb049f0b1dd6ee8ba2a9b58: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260811 02:56:07.996788 2101 consensus_queue.cc:260] T eec61a63b4bf409cbb278b21c26eb829 P 5df3a0a85d0c49f6b582a258e98d025b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0563bdf80aa64bff8b29424496bc9d76" member_type: VOTER last_known_addr { host: "127.12.138.194" port: 45285 } } peers { permanent_uuid: "5df3a0a85d0c49f6b582a258e98d025b" member_type: VOTER last_known_addr { host: "127.12.138.195" port: 39337 } } peers { permanent_uuid: "74701c073eb049f0b1dd6ee8ba2a9b58" member_type: VOTER last_known_addr { host: "127.12.138.193" port: 44263 } }
I20260811 02:56:07.997479 2082 heartbeater.cc:503] Master 127.12.138.254:43779 was elected leader, sending a full tablet report...
I20260811 02:56:07.997927 2101 ts_tablet_manager.cc:1428] T eec61a63b4bf409cbb278b21c26eb829 P 5df3a0a85d0c49f6b582a258e98d025b: Time spent starting tablet: real 0.006s user 0.005s sys 0.001s
I20260811 02:56:08.001482 2100 raft_consensus.cc:348] T eec61a63b4bf409cbb278b21c26eb829 P 0563bdf80aa64bff8b29424496bc9d76 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0563bdf80aa64bff8b29424496bc9d76" member_type: VOTER last_known_addr { host: "127.12.138.194" port: 45285 } } peers { permanent_uuid: "5df3a0a85d0c49f6b582a258e98d025b" member_type: VOTER last_known_addr { host: "127.12.138.195" port: 39337 } } peers { permanent_uuid: "74701c073eb049f0b1dd6ee8ba2a9b58" member_type: VOTER last_known_addr { host: "127.12.138.193" port: 44263 } }
I20260811 02:56:08.001794 2100 raft_consensus.cc:374] T eec61a63b4bf409cbb278b21c26eb829 P 0563bdf80aa64bff8b29424496bc9d76 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 02:56:08.001861 2100 raft_consensus.cc:729] T eec61a63b4bf409cbb278b21c26eb829 P 0563bdf80aa64bff8b29424496bc9d76 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0563bdf80aa64bff8b29424496bc9d76, State: Initialized, Role: FOLLOWER
I20260811 02:56:08.002244 2100 consensus_queue.cc:260] T eec61a63b4bf409cbb278b21c26eb829 P 0563bdf80aa64bff8b29424496bc9d76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0563bdf80aa64bff8b29424496bc9d76" member_type: VOTER last_known_addr { host: "127.12.138.194" port: 45285 } } peers { permanent_uuid: "5df3a0a85d0c49f6b582a258e98d025b" member_type: VOTER last_known_addr { host: "127.12.138.195" port: 39337 } } peers { permanent_uuid: "74701c073eb049f0b1dd6ee8ba2a9b58" member_type: VOTER last_known_addr { host: "127.12.138.193" port: 44263 } }
I20260811 02:56:08.002965 1931 heartbeater.cc:503] Master 127.12.138.254:43779 was elected leader, sending a full tablet report...
I20260811 02:56:08.003105 2100 ts_tablet_manager.cc:1428] T eec61a63b4bf409cbb278b21c26eb829 P 0563bdf80aa64bff8b29424496bc9d76: Time spent starting tablet: real 0.007s user 0.007s sys 0.000s
I20260811 02:56:08.055258 2106 raft_consensus.cc:482] T eec61a63b4bf409cbb278b21c26eb829 P 5df3a0a85d0c49f6b582a258e98d025b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 02:56:08.055560 2106 raft_consensus.cc:504] T eec61a63b4bf409cbb278b21c26eb829 P 5df3a0a85d0c49f6b582a258e98d025b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0563bdf80aa64bff8b29424496bc9d76" member_type: VOTER last_known_addr { host: "127.12.138.194" port: 45285 } } peers { permanent_uuid: "5df3a0a85d0c49f6b582a258e98d025b" member_type: VOTER last_known_addr { host: "127.12.138.195" port: 39337 } } peers { permanent_uuid: "74701c073eb049f0b1dd6ee8ba2a9b58" member_type: VOTER last_known_addr { host: "127.12.138.193" port: 44263 } }
I20260811 02:56:08.056808 2106 leader_election.cc:288] T eec61a63b4bf409cbb278b21c26eb829 P 5df3a0a85d0c49f6b582a258e98d025b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0563bdf80aa64bff8b29424496bc9d76 (127.12.138.194:45285), 74701c073eb049f0b1dd6ee8ba2a9b58 (127.12.138.193:44263)
I20260811 02:56:08.061121 2098 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:56:08.057238 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.138.195:47861 (local address 127.12.138.193:44263)
0811 02:56:08.057361 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0811 02:56:08.057367 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 02:56:08.057821 (+ 454us) server_negotiation.cc:407] Connection header received
0811 02:56:08.057890 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:56:08.057893 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:56:08.057952 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:56:08.058043 (+ 91us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 02:56:08.058445 (+ 402us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:56:08.059047 (+ 602us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:56:08.060166 (+ 1119us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:56:08.060544 (+ 378us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:56:08.060666 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:56:08.060722 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0811 02:56:08.060911 (+ 189us) server_negotiation.cc:299] Negotiation successful
0811 02:56:08.060986 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260811 02:56:08.061136 2084 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:56:08.057055 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.138.194:45285 (local address 127.12.138.195:40121)
0811 02:56:08.057229 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0811 02:56:08.057245 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 02:56:08.057349 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:56:08.057712 (+ 363us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:56:08.057715 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:56:08.057766 (+ 51us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 02:56:08.057972 (+ 206us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:56:08.057984 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:56:08.058945 (+ 961us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:56:08.058948 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:56:08.060025 (+ 1077us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:56:08.060036 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:56:08.060700 (+ 664us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:56:08.060704 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:56:08.060812 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:56:08.060828 (+ 16us) client_negotiation.cc:769] Sending connection context
0811 02:56:08.060922 (+ 94us) client_negotiation.cc:240] Negotiation successful
0811 02:56:08.060986 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":65,"mutex_wait_us":68}
I20260811 02:56:08.061170 2096 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:56:08.057193 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.138.195:40121 (local address 127.12.138.194:45285)
0811 02:56:08.057327 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0811 02:56:08.057331 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 02:56:08.057349 (+ 18us) server_negotiation.cc:407] Connection header received
0811 02:56:08.057507 (+ 158us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:56:08.057512 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:56:08.057576 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:56:08.057653 (+ 77us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 02:56:08.058128 (+ 475us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:56:08.058832 (+ 704us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:56:08.060153 (+ 1321us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:56:08.060558 (+ 405us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:56:08.060689 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:56:08.060743 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0811 02:56:08.060951 (+ 208us) server_negotiation.cc:299] Negotiation successful
0811 02:56:08.061022 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260811 02:56:08.061182 2108 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:56:08.057063 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.138.193:44263 (local address 127.12.138.195:47861)
0811 02:56:08.057590 (+ 527us) negotiation.cc:107] Waiting for socket to connect
0811 02:56:08.057624 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0811 02:56:08.057766 (+ 142us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:56:08.058078 (+ 312us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:56:08.058083 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:56:08.058135 (+ 52us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 02:56:08.058304 (+ 169us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:56:08.058312 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:56:08.059160 (+ 848us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:56:08.059163 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:56:08.060028 (+ 865us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:56:08.060039 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:56:08.060669 (+ 630us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:56:08.060673 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:56:08.060781 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:56:08.060810 (+ 29us) client_negotiation.cc:769] Sending connection context
0811 02:56:08.060873 (+ 63us) client_negotiation.cc:240] Negotiation successful
0811 02:56:08.060959 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":455,"mutex_wait_us":207,"thread_start_us":151,"threads_started":1}
I20260811 02:56:08.061964 1878 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "eec61a63b4bf409cbb278b21c26eb829" candidate_uuid: "5df3a0a85d0c49f6b582a258e98d025b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0563bdf80aa64bff8b29424496bc9d76" is_pre_election: true
I20260811 02:56:08.062247 1724 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "eec61a63b4bf409cbb278b21c26eb829" candidate_uuid: "5df3a0a85d0c49f6b582a258e98d025b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "74701c073eb049f0b1dd6ee8ba2a9b58" is_pre_election: true
I20260811 02:56:08.062310 1878 raft_consensus.cc:2454] T eec61a63b4bf409cbb278b21c26eb829 P 0563bdf80aa64bff8b29424496bc9d76 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5df3a0a85d0c49f6b582a258e98d025b in term 0.
I20260811 02:56:08.062516 1724 raft_consensus.cc:2454] T eec61a63b4bf409cbb278b21c26eb829 P 74701c073eb049f0b1dd6ee8ba2a9b58 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5df3a0a85d0c49f6b582a258e98d025b in term 0.
I20260811 02:56:08.062922 1955 leader_election.cc:302] T eec61a63b4bf409cbb278b21c26eb829 P 5df3a0a85d0c49f6b582a258e98d025b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 0563bdf80aa64bff8b29424496bc9d76, 5df3a0a85d0c49f6b582a258e98d025b, 74701c073eb049f0b1dd6ee8ba2a9b58; no voters:
I20260811 02:56:08.063246 2106 raft_consensus.cc:2781] T eec61a63b4bf409cbb278b21c26eb829 P 5df3a0a85d0c49f6b582a258e98d025b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 02:56:08.063326 2106 raft_consensus.cc:482] T eec61a63b4bf409cbb278b21c26eb829 P 5df3a0a85d0c49f6b582a258e98d025b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 02:56:08.063375 2106 raft_consensus.cc:3037] T eec61a63b4bf409cbb278b21c26eb829 P 5df3a0a85d0c49f6b582a258e98d025b [term 0 FOLLOWER]: Advancing to term 1
I20260811 02:56:08.064641 2106 raft_consensus.cc:504] T eec61a63b4bf409cbb278b21c26eb829 P 5df3a0a85d0c49f6b582a258e98d025b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0563bdf80aa64bff8b29424496bc9d76" member_type: VOTER last_known_addr { host: "127.12.138.194" port: 45285 } } peers { permanent_uuid: "5df3a0a85d0c49f6b582a258e98d025b" member_type: VOTER last_known_addr { host: "127.12.138.195" port: 39337 } } peers { permanent_uuid: "74701c073eb049f0b1dd6ee8ba2a9b58" member_type: VOTER last_known_addr { host: "127.12.138.193" port: 44263 } }
I20260811 02:56:08.065003 2106 leader_election.cc:288] T eec61a63b4bf409cbb278b21c26eb829 P 5df3a0a85d0c49f6b582a258e98d025b [CANDIDATE]: Term 1 election: Requested vote from peers 0563bdf80aa64bff8b29424496bc9d76 (127.12.138.194:45285), 74701c073eb049f0b1dd6ee8ba2a9b58 (127.12.138.193:44263)
I20260811 02:56:08.065451 1724 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "eec61a63b4bf409cbb278b21c26eb829" candidate_uuid: "5df3a0a85d0c49f6b582a258e98d025b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "74701c073eb049f0b1dd6ee8ba2a9b58"
I20260811 02:56:08.065440 1878 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "eec61a63b4bf409cbb278b21c26eb829" candidate_uuid: "5df3a0a85d0c49f6b582a258e98d025b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0563bdf80aa64bff8b29424496bc9d76"
I20260811 02:56:08.065603 1878 raft_consensus.cc:3037] T eec61a63b4bf409cbb278b21c26eb829 P 0563bdf80aa64bff8b29424496bc9d76 [term 0 FOLLOWER]: Advancing to term 1
I20260811 02:56:08.065577 1724 raft_consensus.cc:3037] T eec61a63b4bf409cbb278b21c26eb829 P 74701c073eb049f0b1dd6ee8ba2a9b58 [term 0 FOLLOWER]: Advancing to term 1
I20260811 02:56:08.067013 1878 raft_consensus.cc:2454] T eec61a63b4bf409cbb278b21c26eb829 P 0563bdf80aa64bff8b29424496bc9d76 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5df3a0a85d0c49f6b582a258e98d025b in term 1.
I20260811 02:56:08.067070 1724 raft_consensus.cc:2454] T eec61a63b4bf409cbb278b21c26eb829 P 74701c073eb049f0b1dd6ee8ba2a9b58 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5df3a0a85d0c49f6b582a258e98d025b in term 1.
I20260811 02:56:08.067376 1954 leader_election.cc:302] T eec61a63b4bf409cbb278b21c26eb829 P 5df3a0a85d0c49f6b582a258e98d025b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5df3a0a85d0c49f6b582a258e98d025b, 74701c073eb049f0b1dd6ee8ba2a9b58; no voters:
I20260811 02:56:08.067600 2106 raft_consensus.cc:2781] T eec61a63b4bf409cbb278b21c26eb829 P 5df3a0a85d0c49f6b582a258e98d025b [term 1 FOLLOWER]: Leader election won for term 1
I20260811 02:56:08.067866 2106 raft_consensus.cc:686] T eec61a63b4bf409cbb278b21c26eb829 P 5df3a0a85d0c49f6b582a258e98d025b [term 1 LEADER]: Becoming Leader. State: Replica: 5df3a0a85d0c49f6b582a258e98d025b, State: Running, Role: LEADER
I20260811 02:56:08.068234 2106 consensus_queue.cc:237] T eec61a63b4bf409cbb278b21c26eb829 P 5df3a0a85d0c49f6b582a258e98d025b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0563bdf80aa64bff8b29424496bc9d76" member_type: VOTER last_known_addr { host: "127.12.138.194" port: 45285 } } peers { permanent_uuid: "5df3a0a85d0c49f6b582a258e98d025b" member_type: VOTER last_known_addr { host: "127.12.138.195" port: 39337 } } peers { permanent_uuid: "74701c073eb049f0b1dd6ee8ba2a9b58" member_type: VOTER last_known_addr { host: "127.12.138.193" port: 44263 } }
I20260811 02:56:08.071679 1563 catalog_manager.cc:5495] T eec61a63b4bf409cbb278b21c26eb829 P 5df3a0a85d0c49f6b582a258e98d025b reported cstate change: term changed from 0 to 1, leader changed from <none> to 5df3a0a85d0c49f6b582a258e98d025b (127.12.138.195). New cstate: current_term: 1 leader_uuid: "5df3a0a85d0c49f6b582a258e98d025b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0563bdf80aa64bff8b29424496bc9d76" member_type: VOTER last_known_addr { host: "127.12.138.194" port: 45285 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5df3a0a85d0c49f6b582a258e98d025b" member_type: VOTER last_known_addr { host: "127.12.138.195" port: 39337 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "74701c073eb049f0b1dd6ee8ba2a9b58" member_type: VOTER last_known_addr { host: "127.12.138.193" port: 44263 } health_report { overall_health: UNKNOWN } } }
I20260811 02:56:08.101589 2095 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:56:08.097615 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34704 (local address 127.12.138.195:39337)
0811 02:56:08.097785 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0811 02:56:08.097790 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 02:56:08.097805 (+ 15us) server_negotiation.cc:407] Connection header received
0811 02:56:08.097924 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:56:08.097927 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:56:08.097983 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:56:08.098078 (+ 95us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 02:56:08.098413 (+ 335us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:56:08.098948 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:56:08.099975 (+ 1027us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:56:08.100187 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:56:08.100276 (+ 89us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:56:08.100640 (+ 364us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 02:56:08.101069 (+ 429us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 02:56:08.101212 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0811 02:56:08.101346 (+ 134us) server_negotiation.cc:299] Negotiation successful
0811 02:56:08.101383 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260811 02:56:08.107719 1878 raft_consensus.cc:1261] T eec61a63b4bf409cbb278b21c26eb829 P 0563bdf80aa64bff8b29424496bc9d76 [term 1 FOLLOWER]: Refusing update from remote peer 5df3a0a85d0c49f6b582a258e98d025b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 02:56:08.107717 1724 raft_consensus.cc:1261] T eec61a63b4bf409cbb278b21c26eb829 P 74701c073eb049f0b1dd6ee8ba2a9b58 [term 1 FOLLOWER]: Refusing update from remote peer 5df3a0a85d0c49f6b582a258e98d025b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 02:56:08.108457 2109 consensus_queue.cc:1035] T eec61a63b4bf409cbb278b21c26eb829 P 5df3a0a85d0c49f6b582a258e98d025b [LEADER]: Connected to new peer: Peer: permanent_uuid: "74701c073eb049f0b1dd6ee8ba2a9b58" member_type: VOTER last_known_addr { host: "127.12.138.193" port: 44263 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 02:56:08.108650 2106 consensus_queue.cc:1035] T eec61a63b4bf409cbb278b21c26eb829 P 5df3a0a85d0c49f6b582a258e98d025b [LEADER]: Connected to new peer: Peer: permanent_uuid: "0563bdf80aa64bff8b29424496bc9d76" member_type: VOTER last_known_addr { host: "127.12.138.194" port: 45285 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 02:56:08.116811 2114 mvcc.cc:205] Tried to move back new op lower bound from 7317163901361868800 to 7317163901210472448. Current Snapshot: MvccSnapshot[applied={T|T < 7317163901361868800}]
I20260811 02:56:08.118263 2113 mvcc.cc:205] Tried to move back new op lower bound from 7317163901361868800 to 7317163901210472448. Current Snapshot: MvccSnapshot[applied={T|T < 7317163901361868800}]
I20260811 02:56:08.125795 1561 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33594:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 02:56:08.126010 1561 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33594:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 02:56:08.128901 1561 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 95a1774bfadb4fc0967091217105576d: Sending DeleteTablet for 3 replicas of tablet eec61a63b4bf409cbb278b21c26eb829
I20260811 02:56:08.129832 1855 tablet_service.cc:1514] Processing DeleteTablet for tablet eec61a63b4bf409cbb278b21c26eb829 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 02:56:08 UTC) from {username='kudu', principal='kudu/127.12.138.254@KRBTEST.COM'} at 127.0.0.1:49360
I20260811 02:56:08.129992 1703 tablet_service.cc:1514] Processing DeleteTablet for tablet eec61a63b4bf409cbb278b21c26eb829 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 02:56:08 UTC) from {username='kudu', principal='kudu/127.12.138.254@KRBTEST.COM'} at 127.0.0.1:45614
I20260811 02:56:08.130404 2124 tablet_replica.cc:330] T eec61a63b4bf409cbb278b21c26eb829 P 0563bdf80aa64bff8b29424496bc9d76: stopping tablet replica
I20260811 02:56:08.130810 2124 raft_consensus.cc:2229] T eec61a63b4bf409cbb278b21c26eb829 P 0563bdf80aa64bff8b29424496bc9d76 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 02:56:08.130833 12843 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDawEwS/build/debug/bin/kudu with pid 1633
I20260811 02:56:08.130823 2009 tablet_service.cc:1514] Processing DeleteTablet for tablet eec61a63b4bf409cbb278b21c26eb829 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 02:56:08 UTC) from {username='kudu', principal='kudu/127.12.138.254@KRBTEST.COM'} at 127.0.0.1:34696
I20260811 02:56:08.131304 2124 raft_consensus.cc:2258] T eec61a63b4bf409cbb278b21c26eb829 P 0563bdf80aa64bff8b29424496bc9d76 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 02:56:08.131552 2128 tablet_replica.cc:330] T eec61a63b4bf409cbb278b21c26eb829 P 5df3a0a85d0c49f6b582a258e98d025b: stopping tablet replica
I20260811 02:56:08.131950 2128 raft_consensus.cc:2229] T eec61a63b4bf409cbb278b21c26eb829 P 5df3a0a85d0c49f6b582a258e98d025b [term 1 LEADER]: Raft consensus shutting down.
I20260811 02:56:08.132278 2128 raft_consensus.cc:2258] T eec61a63b4bf409cbb278b21c26eb829 P 5df3a0a85d0c49f6b582a258e98d025b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 02:56:08.134199 2124 ts_tablet_manager.cc:1905] T eec61a63b4bf409cbb278b21c26eb829 P 0563bdf80aa64bff8b29424496bc9d76: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 02:56:08.134199 2128 ts_tablet_manager.cc:1905] T eec61a63b4bf409cbb278b21c26eb829 P 5df3a0a85d0c49f6b582a258e98d025b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 02:56:08.136989 2124 ts_tablet_manager.cc:1918] T eec61a63b4bf409cbb278b21c26eb829 P 0563bdf80aa64bff8b29424496bc9d76: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 02:56:08.136989 2128 ts_tablet_manager.cc:1918] T eec61a63b4bf409cbb278b21c26eb829 P 5df3a0a85d0c49f6b582a258e98d025b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 02:56:08.137094 2128 log.cc:1196] T eec61a63b4bf409cbb278b21c26eb829 P 5df3a0a85d0c49f6b582a258e98d025b: Deleting WAL directory at /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786416707705494-12843-0/minicluster-data/ts-2/wal/wals/eec61a63b4bf409cbb278b21c26eb829
I20260811 02:56:08.137259 2124 log.cc:1196] T eec61a63b4bf409cbb278b21c26eb829 P 0563bdf80aa64bff8b29424496bc9d76: Deleting WAL directory at /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786416707705494-12843-0/minicluster-data/ts-1/wal/wals/eec61a63b4bf409cbb278b21c26eb829
I20260811 02:56:08.137485 2128 ts_tablet_manager.cc:1939] T eec61a63b4bf409cbb278b21c26eb829 P 5df3a0a85d0c49f6b582a258e98d025b: Deleting consensus metadata
I20260811 02:56:08.137744 2124 ts_tablet_manager.cc:1939] T eec61a63b4bf409cbb278b21c26eb829 P 0563bdf80aa64bff8b29424496bc9d76: Deleting consensus metadata
I20260811 02:56:08.138770 1547 catalog_manager.cc:4841] TS 0563bdf80aa64bff8b29424496bc9d76 (127.12.138.194:45285): tablet eec61a63b4bf409cbb278b21c26eb829 (table test-table [id=3586fb46aa7a4028a36840c88a0545d3]) successfully deleted
I20260811 02:56:08.138751 1548 catalog_manager.cc:4841] TS 5df3a0a85d0c49f6b582a258e98d025b (127.12.138.195:39337): tablet eec61a63b4bf409cbb278b21c26eb829 (table test-table [id=3586fb46aa7a4028a36840c88a0545d3]) successfully deleted
W20260811 02:56:08.145707 1546 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.12.138.193:44263 (error 108)
I20260811 02:56:08.145816 12843 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDawEwS/build/debug/bin/kudu with pid 1781
I20260811 02:56:08.157122 2093 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:56:08.156796 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.138.193:44263 (local address 127.0.0.1:45618)
0811 02:56:08.156946 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0811 02:56:08.157034 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.138.193:44263: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
W20260811 02:56:08.157423 1546 catalog_manager.cc:4584] TS 74701c073eb049f0b1dd6ee8ba2a9b58 (127.12.138.193:44263): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet eec61a63b4bf409cbb278b21c26eb829: Network error: Client connection negotiation failed: client connection to 127.12.138.193:44263: connect: Connection refused (error 111)
I20260811 02:56:08.163843 12843 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDawEwS/build/debug/bin/kudu with pid 1936
I20260811 02:56:08.173329 12843 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDawEwS/build/debug/bin/kudu with pid 1529
2026-08-11T02:56:08Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/2 (2530 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/3
2026-08-11T02:56:08Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T02:56:08Z Disabled control of system clock
I20260811 02:56:08.207021 12843 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDawEwS/build/debug/bin/kudu
/tmp/dist-test-taskDawEwS/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786416707705494-12843-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786416707705494-12843-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786416707705494-12843-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786416707705494-12843-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.138.254:36093
--webserver_interface=127.12.138.254
--webserver_port=0
--builtin_ntp_servers=127.12.138.212:45925
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.12.138.254:36093
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260811 02:56:08.329844 2135 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 02:56:08.330155 2135 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 02:56:08.330194 2135 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 02:56:08.334265 2135 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 02:56:08.334328 2135 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 02:56:08.334347 2135 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 02:56:08.334367 2135 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 02:56:08.334386 2135 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 02:56:08.339149 2135 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.138.212:45925
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786416707705494-12843-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786416707705494-12843-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.12.138.254:36093
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.138.254:36093
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786416707705494-12843-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.12.138.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2135
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786416707705494-12843-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 02:09:49 UTC on ccycloud.quasar-fsnqqc.root.comops.site
build id 4579570
I20260811 02:56:08.340204 2135 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 02:56:08.341583 2135 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 02:56:08.350944 2143 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:56:08.351092 2135 server_base.cc:1034] running on GCE node
W20260811 02:56:08.350916 2140 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:56:08.350965 2141 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:56:08.351881 2135 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 02:56:08.353106 2135 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 02:56:08.354314 2135 hybrid_clock.cc:648] HybridClock initialized: now 1786416968354301 us; error 29 us; skew 500 ppm
I20260811 02:56:08.356331 2135 webserver.cc:460] Webserver started at http://127.12.138.254:41263/ using document root <none> and password file <none>
I20260811 02:56:08.356895 2135 fs_manager.cc:359] Metadata directory not provided
I20260811 02:56:08.356959 2135 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 02:56:08.357131 2135 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 02:56:08.359290 2135 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786416707705494-12843-0/minicluster-data/master-0/data/instance:
uuid: "a9f86b8a3ab6415b8e77a08de084718e"
format_stamp: "Formatted at 2026-08-11 02:56:08 on dist-test-slave-9gcw"
server_key: "c85a19c6bf3abb7134d876020746b42f"
server_key_iv: "3aef516d768ecf0698b83c579621bb35"
server_key_version: "encryptionkey@0"
I20260811 02:56:08.359831 2135 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786416707705494-12843-0/minicluster-data/master-0/wal/instance:
uuid: "a9f86b8a3ab6415b8e77a08de084718e"
format_stamp: "Formatted at 2026-08-11 02:56:08 on dist-test-slave-9gcw"
server_key: "c85a19c6bf3abb7134d876020746b42f"
server_key_iv: "3aef516d768ecf0698b83c579621bb35"
server_key_version: "encryptionkey@0"
I20260811 02:56:08.363773 2135 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.001s sys 0.004s
I20260811 02:56:08.366279 2150 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 02:56:08.367530 2135 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260811 02:56:08.367643 2135 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786416707705494-12843-0/minicluster-data/master-0/data,/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786416707705494-12843-0/minicluster-data/master-0/wal
uuid: "a9f86b8a3ab6415b8e77a08de084718e"
format_stamp: "Formatted at 2026-08-11 02:56:08 on dist-test-slave-9gcw"
server_key: "c85a19c6bf3abb7134d876020746b42f"
server_key_iv: "3aef516d768ecf0698b83c579621bb35"
server_key_version: "encryptionkey@0"
I20260811 02:56:08.367780 2135 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786416707705494-12843-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786416707705494-12843-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786416707705494-12843-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 02:56:08.383471 2135 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 02:56:08.384218 2135 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 02:56:08.384426 2135 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 02:56:08.392112 2135 rpc_server.cc:307] RPC server started. Bound to: 127.12.138.254:36093
I20260811 02:56:08.392158 2209 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.138.254:36093 every 8 connection(s)
I20260811 02:56:08.393261 2135 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786416707705494-12843-0/minicluster-data/master-0/data/info.pb
I20260811 02:56:08.393877 12843 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDawEwS/build/debug/bin/kudu as pid 2135
I20260811 02:56:08.394027 12843 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786416707705494-12843-0/minicluster-data/master-0/wal/instance
I20260811 02:56:08.394340 12843 external_mini_cluster.cc:1428] Setting key e27033ec9510915b1ef25c282d6c9e05
I20260811 02:56:08.397127 2210 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 02:56:08.403012 2210 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a9f86b8a3ab6415b8e77a08de084718e: Bootstrap starting.
I20260811 02:56:08.403940 2214 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:56:08.395851 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46872 (local address 127.12.138.254:36093)
0811 02:56:08.396558 (+ 707us) server_negotiation.cc:206] Beginning negotiation
0811 02:56:08.396574 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0811 02:56:08.396611 (+ 37us) server_negotiation.cc:407] Connection header received
0811 02:56:08.397478 (+ 867us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:56:08.397497 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:56:08.397896 (+ 399us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:56:08.398240 (+ 344us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:56:08.398685 (+ 445us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:56:08.399646 (+ 961us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:56:08.400545 (+ 899us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:56:08.401027 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:56:08.401249 (+ 222us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:56:08.402132 (+ 883us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:56:08.402137 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:56:08.402150 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 02:56:08.402560 (+ 410us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:56:08.402615 (+ 55us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 02:56:08.402636 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:56:08.402747 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0811 02:56:08.402924 (+ 177us) server_negotiation.cc:299] Negotiation successful
0811 02:56:08.403225 (+ 301us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":335,"thread_start_us":158,"threads_started":1}
I20260811 02:56:08.405601 2210 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a9f86b8a3ab6415b8e77a08de084718e: Neither blocks nor log segments found. Creating new log.
I20260811 02:56:08.406530 2210 log.cc:824] T 00000000000000000000000000000000 P a9f86b8a3ab6415b8e77a08de084718e: Log is configured to *not* fsync() on all Append() calls
I20260811 02:56:08.408509 2210 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a9f86b8a3ab6415b8e77a08de084718e: No bootstrap required, opened a new log
I20260811 02:56:08.412619 2210 raft_consensus.cc:348] T 00000000000000000000000000000000 P a9f86b8a3ab6415b8e77a08de084718e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a9f86b8a3ab6415b8e77a08de084718e" member_type: VOTER last_known_addr { host: "127.12.138.254" port: 36093 } }
I20260811 02:56:08.412910 2210 raft_consensus.cc:374] T 00000000000000000000000000000000 P a9f86b8a3ab6415b8e77a08de084718e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 02:56:08.412961 2210 raft_consensus.cc:729] T 00000000000000000000000000000000 P a9f86b8a3ab6415b8e77a08de084718e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a9f86b8a3ab6415b8e77a08de084718e, State: Initialized, Role: FOLLOWER
I20260811 02:56:08.413419 2210 consensus_queue.cc:260] T 00000000000000000000000000000000 P a9f86b8a3ab6415b8e77a08de084718e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a9f86b8a3ab6415b8e77a08de084718e" member_type: VOTER last_known_addr { host: "127.12.138.254" port: 36093 } }
I20260811 02:56:08.413549 2210 raft_consensus.cc:388] T 00000000000000000000000000000000 P a9f86b8a3ab6415b8e77a08de084718e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 02:56:08.413596 2210 raft_consensus.cc:482] T 00000000000000000000000000000000 P a9f86b8a3ab6415b8e77a08de084718e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 02:56:08.413673 2210 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a9f86b8a3ab6415b8e77a08de084718e [term 0 FOLLOWER]: Advancing to term 1
I20260811 02:56:08.414620 2210 raft_consensus.cc:504] T 00000000000000000000000000000000 P a9f86b8a3ab6415b8e77a08de084718e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a9f86b8a3ab6415b8e77a08de084718e" member_type: VOTER last_known_addr { host: "127.12.138.254" port: 36093 } }
I20260811 02:56:08.414942 2210 leader_election.cc:302] T 00000000000000000000000000000000 P a9f86b8a3ab6415b8e77a08de084718e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a9f86b8a3ab6415b8e77a08de084718e; no voters:
I20260811 02:56:08.415261 2210 leader_election.cc:288] T 00000000000000000000000000000000 P a9f86b8a3ab6415b8e77a08de084718e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 02:56:08.415385 2216 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a9f86b8a3ab6415b8e77a08de084718e [term 1 FOLLOWER]: Leader election won for term 1
I20260811 02:56:08.415656 2216 raft_consensus.cc:686] T 00000000000000000000000000000000 P a9f86b8a3ab6415b8e77a08de084718e [term 1 LEADER]: Becoming Leader. State: Replica: a9f86b8a3ab6415b8e77a08de084718e, State: Running, Role: LEADER
I20260811 02:56:08.415972 2216 consensus_queue.cc:237] T 00000000000000000000000000000000 P a9f86b8a3ab6415b8e77a08de084718e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a9f86b8a3ab6415b8e77a08de084718e" member_type: VOTER last_known_addr { host: "127.12.138.254" port: 36093 } }
I20260811 02:56:08.416543 2210 sys_catalog.cc:564] T 00000000000000000000000000000000 P a9f86b8a3ab6415b8e77a08de084718e [sys.catalog]: configured and running, proceeding with master startup.
I20260811 02:56:08.417699 2218 sys_catalog.cc:455] T 00000000000000000000000000000000 P a9f86b8a3ab6415b8e77a08de084718e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a9f86b8a3ab6415b8e77a08de084718e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a9f86b8a3ab6415b8e77a08de084718e" member_type: VOTER last_known_addr { host: "127.12.138.254" port: 36093 } } }
I20260811 02:56:08.417860 2218 sys_catalog.cc:458] T 00000000000000000000000000000000 P a9f86b8a3ab6415b8e77a08de084718e [sys.catalog]: This master's current role is: LEADER
I20260811 02:56:08.418303 2227 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 02:56:08.418843 2219 sys_catalog.cc:455] T 00000000000000000000000000000000 P a9f86b8a3ab6415b8e77a08de084718e [sys.catalog]: SysCatalogTable state changed. Reason: New leader a9f86b8a3ab6415b8e77a08de084718e. Latest consensus state: current_term: 1 leader_uuid: "a9f86b8a3ab6415b8e77a08de084718e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a9f86b8a3ab6415b8e77a08de084718e" member_type: VOTER last_known_addr { host: "127.12.138.254" port: 36093 } } }
I20260811 02:56:08.418953 2219 sys_catalog.cc:458] T 00000000000000000000000000000000 P a9f86b8a3ab6415b8e77a08de084718e [sys.catalog]: This master's current role is: LEADER
I20260811 02:56:08.421757 2227 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 02:56:08.427767 2227 catalog_manager.cc:1347] Generated new cluster ID: 2061d5307784489fa2c452276d9180b0
I20260811 02:56:08.427840 2227 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 02:56:08.440866 2227 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 02:56:08.442198 2227 catalog_manager.cc:1504] Loading token signing keys...
I20260811 02:56:08.464548 2227 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a9f86b8a3ab6415b8e77a08de084718e: Generated new TSK 0
I20260811 02:56:08.465585 2227 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 02:56:08.480829 12843 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDawEwS/build/debug/bin/kudu
/tmp/dist-test-taskDawEwS/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786416707705494-12843-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786416707705494-12843-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786416707705494-12843-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786416707705494-12843-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.138.193:0
--local_ip_for_outbound_sockets=127.12.138.193
--webserver_interface=127.12.138.193
--webserver_port=0
--tserver_master_addrs=127.12.138.254:36093
--builtin_ntp_servers=127.12.138.212:45925
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260811 02:56:08.599576 2237 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 02:56:08.599853 2237 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 02:56:08.599892 2237 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 02:56:08.603730 2237 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 02:56:08.603781 2237 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 02:56:08.603833 2237 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.138.193
I20260811 02:56:08.608440 2237 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.138.212:45925
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786416707705494-12843-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786416707705494-12843-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.138.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786416707705494-12843-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.12.138.193
--webserver_port=0
--tserver_master_addrs=127.12.138.254:36093
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2237
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.138.193
--log_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786416707705494-12843-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 02:09:49 UTC on ccycloud.quasar-fsnqqc.root.comops.site
build id 4579570
I20260811 02:56:08.609483 2237 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 02:56:08.610802 2237 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 02:56:08.621194 2246 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:56:08.621356 2237 server_base.cc:1034] running on GCE node
W20260811 02:56:08.621593 2244 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:56:08.621577 2243 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:56:08.622133 2237 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 02:56:08.622805 2237 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 02:56:08.623996 2237 hybrid_clock.cc:648] HybridClock initialized: now 1786416968623975 us; error 37 us; skew 500 ppm
I20260811 02:56:08.626096 2237 webserver.cc:460] Webserver started at http://127.12.138.193:40419/ using document root <none> and password file <none>
I20260811 02:56:08.626722 2237 fs_manager.cc:359] Metadata directory not provided
I20260811 02:56:08.626819 2237 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 02:56:08.627064 2237 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 02:56:08.629333 2237 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786416707705494-12843-0/minicluster-data/ts-0/data/instance:
uuid: "71b281515e0e416383d3224578671456"
format_stamp: "Formatted at 2026-08-11 02:56:08 on dist-test-slave-9gcw"
server_key: "1cf3a6cfd36cd2ba8efd2bbb8ed3e28b"
server_key_iv: "d7c909b658765815fbf9ab2bbddd644f"
server_key_version: "encryptionkey@0"
I20260811 02:56:08.629954 2237 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786416707705494-12843-0/minicluster-data/ts-0/wal/instance:
uuid: "71b281515e0e416383d3224578671456"
format_stamp: "Formatted at 2026-08-11 02:56:08 on dist-test-slave-9gcw"
server_key: "1cf3a6cfd36cd2ba8efd2bbb8ed3e28b"
server_key_iv: "d7c909b658765815fbf9ab2bbddd644f"
server_key_version: "encryptionkey@0"
I20260811 02:56:08.633903 2237 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.000s sys 0.003s
I20260811 02:56:08.636552 2252 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 02:56:08.637909 2237 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260811 02:56:08.638036 2237 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786416707705494-12843-0/minicluster-data/ts-0/data,/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786416707705494-12843-0/minicluster-data/ts-0/wal
uuid: "71b281515e0e416383d3224578671456"
format_stamp: "Formatted at 2026-08-11 02:56:08 on dist-test-slave-9gcw"
server_key: "1cf3a6cfd36cd2ba8efd2bbb8ed3e28b"
server_key_iv: "d7c909b658765815fbf9ab2bbddd644f"
server_key_version: "encryptionkey@0"
I20260811 02:56:08.638171 2237 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786416707705494-12843-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786416707705494-12843-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786416707705494-12843-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 02:56:08.650417 2237 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 02:56:08.651068 2237 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 02:56:08.651232 2237 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 02:56:08.651767 2237 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 02:56:08.652695 2237 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 02:56:08.652745 2237 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 02:56:08.652783 2237 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 02:56:08.652808 2237 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 02:56:08.662561 2237 rpc_server.cc:307] RPC server started. Bound to: 127.12.138.193:45423
I20260811 02:56:08.662592 2378 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.138.193:45423 every 8 connection(s)
I20260811 02:56:08.663503 2237 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786416707705494-12843-0/minicluster-data/ts-0/data/info.pb
I20260811 02:56:08.668941 12843 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDawEwS/build/debug/bin/kudu as pid 2237
I20260811 02:56:08.669052 12843 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786416707705494-12843-0/minicluster-data/ts-0/wal/instance
I20260811 02:56:08.669394 12843 external_mini_cluster.cc:1428] Setting key 36d98ce5f946f890a4d70191a4f9c8a1
I20260811 02:56:08.671511 12843 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDawEwS/build/debug/bin/kudu
/tmp/dist-test-taskDawEwS/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786416707705494-12843-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786416707705494-12843-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786416707705494-12843-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786416707705494-12843-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.138.194:0
--local_ip_for_outbound_sockets=127.12.138.194
--webserver_interface=127.12.138.194
--webserver_port=0
--tserver_master_addrs=127.12.138.254:36093
--builtin_ntp_servers=127.12.138.212:45925
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260811 02:56:08.674031 2214 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:56:08.665500 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.138.193:37967 (local address 127.12.138.254:36093)
0811 02:56:08.665729 (+ 229us) server_negotiation.cc:206] Beginning negotiation
0811 02:56:08.665733 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 02:56:08.666913 (+ 1180us) server_negotiation.cc:407] Connection header received
0811 02:56:08.668567 (+ 1654us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:56:08.668570 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:56:08.668647 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:56:08.668751 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:56:08.669824 (+ 1073us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:56:08.670456 (+ 632us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:56:08.671863 (+ 1407us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:56:08.672147 (+ 284us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:56:08.672265 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:56:08.673497 (+ 1232us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:56:08.673501 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:56:08.673504 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 02:56:08.673519 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:56:08.673546 (+ 27us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 02:56:08.673554 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:56:08.673630 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0811 02:56:08.673797 (+ 167us) server_negotiation.cc:299] Negotiation successful
0811 02:56:08.673886 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260811 02:56:08.674947 2381 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:56:08.665913 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.138.254:36093 (local address 127.12.138.193:37967)
0811 02:56:08.666737 (+ 824us) negotiation.cc:107] Waiting for socket to connect
0811 02:56:08.666798 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0811 02:56:08.668346 (+ 1548us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:56:08.668919 (+ 573us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:56:08.668926 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:56:08.669037 (+ 111us) client_negotiation.cc:189] Negotiated authn=SASL
0811 02:56:08.669674 (+ 637us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:56:08.669691 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:56:08.670595 (+ 904us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:56:08.670598 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:56:08.671730 (+ 1132us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:56:08.671746 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:56:08.672307 (+ 561us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:56:08.672312 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:56:08.672487 (+ 175us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:56:08.673206 (+ 719us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 02:56:08.673208 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0811 02:56:08.673259 (+ 51us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 02:56:08.673325 (+ 66us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 02:56:08.673632 (+ 307us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 02:56:08.673645 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 02:56:08.673652 (+ 7us) client_negotiation.cc:769] Sending connection context
0811 02:56:08.673797 (+ 145us) client_negotiation.cc:240] Negotiation successful
0811 02:56:08.674083 (+ 286us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":286,"thread_start_us":109,"threads_started":1}
I20260811 02:56:08.676419 2379 heartbeater.cc:347] Connected to a master server at 127.12.138.254:36093
I20260811 02:56:08.676672 2379 heartbeater.cc:464] Registering TS with master...
I20260811 02:56:08.677309 2379 heartbeater.cc:511] Master 127.12.138.254:36093 requested a full tablet report, sending...
I20260811 02:56:08.679081 2169 ts_manager.cc:194] Registered new tserver with Master: 71b281515e0e416383d3224578671456 (127.12.138.193:45423)
I20260811 02:56:08.680889 2169 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.12.138.193:37967
W20260811 02:56:08.788023 2383 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 02:56:08.788329 2383 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 02:56:08.788369 2383 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 02:56:08.792407 2383 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 02:56:08.792479 2383 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 02:56:08.792534 2383 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.138.194
I20260811 02:56:08.797495 2383 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.138.212:45925
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786416707705494-12843-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786416707705494-12843-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.138.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786416707705494-12843-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.12.138.194
--webserver_port=0
--tserver_master_addrs=127.12.138.254:36093
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2383
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.138.194
--log_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786416707705494-12843-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 02:09:49 UTC on ccycloud.quasar-fsnqqc.root.comops.site
build id 4579570
I20260811 02:56:08.798592 2383 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 02:56:08.799988 2383 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 02:56:08.809928 2389 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:56:08.809973 2391 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:56:08.809966 2383 server_base.cc:1034] running on GCE node
W20260811 02:56:08.809937 2388 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:56:08.810609 2383 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 02:56:08.811280 2383 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 02:56:08.812459 2383 hybrid_clock.cc:648] HybridClock initialized: now 1786416968812444 us; error 35 us; skew 500 ppm
I20260811 02:56:08.814474 2383 webserver.cc:460] Webserver started at http://127.12.138.194:42625/ using document root <none> and password file <none>
I20260811 02:56:08.815156 2383 fs_manager.cc:359] Metadata directory not provided
I20260811 02:56:08.815253 2383 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 02:56:08.815549 2383 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 02:56:08.817836 2383 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786416707705494-12843-0/minicluster-data/ts-1/data/instance:
uuid: "7274df7aa6204213b47ea681f9523e53"
format_stamp: "Formatted at 2026-08-11 02:56:08 on dist-test-slave-9gcw"
server_key: "9a8e6352bc90a4ab5c9b83c278b56a4e"
server_key_iv: "3437087bbf17c1a9e70928d8d04dd6de"
server_key_version: "encryptionkey@0"
I20260811 02:56:08.818492 2383 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786416707705494-12843-0/minicluster-data/ts-1/wal/instance:
uuid: "7274df7aa6204213b47ea681f9523e53"
format_stamp: "Formatted at 2026-08-11 02:56:08 on dist-test-slave-9gcw"
server_key: "9a8e6352bc90a4ab5c9b83c278b56a4e"
server_key_iv: "3437087bbf17c1a9e70928d8d04dd6de"
server_key_version: "encryptionkey@0"
I20260811 02:56:08.822292 2383 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.000s sys 0.004s
I20260811 02:56:08.824913 2398 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 02:56:08.826277 2383 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260811 02:56:08.826423 2383 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786416707705494-12843-0/minicluster-data/ts-1/data,/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786416707705494-12843-0/minicluster-data/ts-1/wal
uuid: "7274df7aa6204213b47ea681f9523e53"
format_stamp: "Formatted at 2026-08-11 02:56:08 on dist-test-slave-9gcw"
server_key: "9a8e6352bc90a4ab5c9b83c278b56a4e"
server_key_iv: "3437087bbf17c1a9e70928d8d04dd6de"
server_key_version: "encryptionkey@0"
I20260811 02:56:08.826537 2383 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786416707705494-12843-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786416707705494-12843-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786416707705494-12843-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 02:56:08.838022 2383 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 02:56:08.838668 2383 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 02:56:08.838872 2383 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 02:56:08.839453 2383 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 02:56:08.840397 2383 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 02:56:08.840469 2383 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 02:56:08.840538 2383 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 02:56:08.840586 2383 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 02:56:08.849936 2383 rpc_server.cc:307] RPC server started. Bound to: 127.12.138.194:46571
I20260811 02:56:08.850013 2529 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.138.194:46571 every 8 connection(s)
I20260811 02:56:08.850900 2383 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786416707705494-12843-0/minicluster-data/ts-1/data/info.pb
I20260811 02:56:08.859618 12843 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDawEwS/build/debug/bin/kudu as pid 2383
I20260811 02:56:08.859791 12843 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786416707705494-12843-0/minicluster-data/ts-1/wal/instance
I20260811 02:56:08.860169 12843 external_mini_cluster.cc:1428] Setting key b0a4497896ba8e8176b1a9e8529f4064
I20260811 02:56:08.860886 2214 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:56:08.852811 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.138.194:54737 (local address 127.12.138.254:36093)
0811 02:56:08.852974 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0811 02:56:08.852978 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 02:56:08.854253 (+ 1275us) server_negotiation.cc:407] Connection header received
0811 02:56:08.855645 (+ 1392us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:56:08.855648 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:56:08.855697 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:56:08.855784 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:56:08.856805 (+ 1021us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:56:08.857434 (+ 629us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:56:08.858653 (+ 1219us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:56:08.858874 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:56:08.858954 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:56:08.860288 (+ 1334us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:56:08.860292 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:56:08.860294 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 02:56:08.860308 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:56:08.860326 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 02:56:08.860334 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:56:08.860393 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0811 02:56:08.860583 (+ 190us) server_negotiation.cc:299] Negotiation successful
0811 02:56:08.860661 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260811 02:56:08.861862 2532 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:56:08.853322 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.138.254:36093 (local address 127.12.138.194:54737)
0811 02:56:08.854083 (+ 761us) negotiation.cc:107] Waiting for socket to connect
0811 02:56:08.854126 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0811 02:56:08.855451 (+ 1325us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:56:08.855961 (+ 510us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:56:08.855967 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:56:08.856052 (+ 85us) client_negotiation.cc:189] Negotiated authn=SASL
0811 02:56:08.856563 (+ 511us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:56:08.856575 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:56:08.857561 (+ 986us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:56:08.857564 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:56:08.858544 (+ 980us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:56:08.858554 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:56:08.859016 (+ 462us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:56:08.859019 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:56:08.859181 (+ 162us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:56:08.859962 (+ 781us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 02:56:08.859965 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0811 02:56:08.860024 (+ 59us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 02:56:08.860108 (+ 84us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 02:56:08.860418 (+ 310us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 02:56:08.860428 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 02:56:08.860435 (+ 7us) client_negotiation.cc:769] Sending connection context
0811 02:56:08.860569 (+ 134us) client_negotiation.cc:240] Negotiation successful
0811 02:56:08.860857 (+ 288us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":364,"thread_start_us":119,"threads_started":1}
I20260811 02:56:08.862495 12843 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDawEwS/build/debug/bin/kudu
/tmp/dist-test-taskDawEwS/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786416707705494-12843-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786416707705494-12843-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786416707705494-12843-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786416707705494-12843-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.138.195:0
--local_ip_for_outbound_sockets=127.12.138.195
--webserver_interface=127.12.138.195
--webserver_port=0
--tserver_master_addrs=127.12.138.254:36093
--builtin_ntp_servers=127.12.138.212:45925
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260811 02:56:08.863016 2530 heartbeater.cc:347] Connected to a master server at 127.12.138.254:36093
I20260811 02:56:08.863277 2530 heartbeater.cc:464] Registering TS with master...
I20260811 02:56:08.863823 2530 heartbeater.cc:511] Master 127.12.138.254:36093 requested a full tablet report, sending...
I20260811 02:56:08.864972 2169 ts_manager.cc:194] Registered new tserver with Master: 7274df7aa6204213b47ea681f9523e53 (127.12.138.194:46571)
I20260811 02:56:08.865602 2169 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.12.138.194:54737
W20260811 02:56:08.984539 2533 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 02:56:08.984820 2533 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 02:56:08.984859 2533 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 02:56:08.988930 2533 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 02:56:08.988999 2533 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 02:56:08.989054 2533 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.138.195
I20260811 02:56:08.993899 2533 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.138.212:45925
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786416707705494-12843-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786416707705494-12843-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.138.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786416707705494-12843-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.12.138.195
--webserver_port=0
--tserver_master_addrs=127.12.138.254:36093
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2533
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.138.195
--log_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786416707705494-12843-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 02:09:49 UTC on ccycloud.quasar-fsnqqc.root.comops.site
build id 4579570
I20260811 02:56:08.995051 2533 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 02:56:08.996567 2533 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 02:56:09.007037 2543 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:56:09.007076 2545 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:56:09.007144 2533 server_base.cc:1034] running on GCE node
W20260811 02:56:09.006994 2540 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:56:09.007647 2533 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 02:56:09.008322 2533 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 02:56:09.009507 2533 hybrid_clock.cc:648] HybridClock initialized: now 1786416969009485 us; error 37 us; skew 500 ppm
I20260811 02:56:09.011564 2533 webserver.cc:460] Webserver started at http://127.12.138.195:44891/ using document root <none> and password file <none>
I20260811 02:56:09.012184 2533 fs_manager.cc:359] Metadata directory not provided
I20260811 02:56:09.012282 2533 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 02:56:09.012523 2533 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 02:56:09.014813 2533 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786416707705494-12843-0/minicluster-data/ts-2/data/instance:
uuid: "4f837d1fb2dc4e89b51025d6a8bd16e3"
format_stamp: "Formatted at 2026-08-11 02:56:09 on dist-test-slave-9gcw"
server_key: "ee0a8f2115989f5b635174fd98c02b37"
server_key_iv: "46b050381b084b72af712691fab70e16"
server_key_version: "encryptionkey@0"
I20260811 02:56:09.015434 2533 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786416707705494-12843-0/minicluster-data/ts-2/wal/instance:
uuid: "4f837d1fb2dc4e89b51025d6a8bd16e3"
format_stamp: "Formatted at 2026-08-11 02:56:09 on dist-test-slave-9gcw"
server_key: "ee0a8f2115989f5b635174fd98c02b37"
server_key_iv: "46b050381b084b72af712691fab70e16"
server_key_version: "encryptionkey@0"
I20260811 02:56:09.019604 2533 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260811 02:56:09.022286 2552 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 02:56:09.023617 2533 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 02:56:09.023792 2533 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786416707705494-12843-0/minicluster-data/ts-2/data,/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786416707705494-12843-0/minicluster-data/ts-2/wal
uuid: "4f837d1fb2dc4e89b51025d6a8bd16e3"
format_stamp: "Formatted at 2026-08-11 02:56:09 on dist-test-slave-9gcw"
server_key: "ee0a8f2115989f5b635174fd98c02b37"
server_key_iv: "46b050381b084b72af712691fab70e16"
server_key_version: "encryptionkey@0"
I20260811 02:56:09.023914 2533 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786416707705494-12843-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786416707705494-12843-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786416707705494-12843-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 02:56:09.036430 2533 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 02:56:09.037269 2533 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 02:56:09.037483 2533 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 02:56:09.038115 2533 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 02:56:09.039147 2533 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 02:56:09.039219 2533 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 02:56:09.039292 2533 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 02:56:09.039333 2533 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 02:56:09.048950 2533 rpc_server.cc:307] RPC server started. Bound to: 127.12.138.195:41525
I20260811 02:56:09.048981 2678 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.138.195:41525 every 8 connection(s)
I20260811 02:56:09.049955 2533 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786416707705494-12843-0/minicluster-data/ts-2/data/info.pb
I20260811 02:56:09.050967 12843 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDawEwS/build/debug/bin/kudu as pid 2533
I20260811 02:56:09.051083 12843 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786416707705494-12843-0/minicluster-data/ts-2/wal/instance
I20260811 02:56:09.051394 12843 external_mini_cluster.cc:1428] Setting key c420a50b3fb2b571497b5ed7b2ea011d
I20260811 02:56:09.060190 2214 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:56:09.052058 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.138.195:43825 (local address 127.12.138.254:36093)
0811 02:56:09.052189 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0811 02:56:09.052192 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 02:56:09.053547 (+ 1355us) server_negotiation.cc:407] Connection header received
0811 02:56:09.055051 (+ 1504us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:56:09.055054 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:56:09.055100 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:56:09.055194 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:56:09.056125 (+ 931us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:56:09.056732 (+ 607us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:56:09.058176 (+ 1444us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:56:09.058396 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:56:09.058483 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:56:09.059763 (+ 1280us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:56:09.059766 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:56:09.059768 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 02:56:09.059781 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:56:09.059801 (+ 20us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 02:56:09.059808 (+ 7us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:56:09.059869 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0811 02:56:09.060008 (+ 139us) server_negotiation.cc:299] Negotiation successful
0811 02:56:09.060063 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260811 02:56:09.061228 2681 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:56:09.052598 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.138.254:36093 (local address 127.12.138.195:43825)
0811 02:56:09.053401 (+ 803us) negotiation.cc:107] Waiting for socket to connect
0811 02:56:09.053461 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0811 02:56:09.054838 (+ 1377us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:56:09.055335 (+ 497us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:56:09.055352 (+ 17us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:56:09.055441 (+ 89us) client_negotiation.cc:189] Negotiated authn=SASL
0811 02:56:09.055966 (+ 525us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:56:09.055979 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:56:09.056878 (+ 899us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:56:09.056881 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:56:09.058039 (+ 1158us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:56:09.058050 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:56:09.058554 (+ 504us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:56:09.058558 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:56:09.058793 (+ 235us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:56:09.059437 (+ 644us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 02:56:09.059439 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0811 02:56:09.059478 (+ 39us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 02:56:09.059543 (+ 65us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 02:56:09.059890 (+ 347us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 02:56:09.059896 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 02:56:09.059901 (+ 5us) client_negotiation.cc:769] Sending connection context
0811 02:56:09.059994 (+ 93us) client_negotiation.cc:240] Negotiation successful
0811 02:56:09.060218 (+ 224us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":318,"thread_start_us":161,"threads_started":1}
I20260811 02:56:09.062477 2679 heartbeater.cc:347] Connected to a master server at 127.12.138.254:36093
I20260811 02:56:09.062712 2679 heartbeater.cc:464] Registering TS with master...
I20260811 02:56:09.063239 2679 heartbeater.cc:511] Master 127.12.138.254:36093 requested a full tablet report, sending...
I20260811 02:56:09.064294 2169 ts_manager.cc:194] Registered new tserver with Master: 4f837d1fb2dc4e89b51025d6a8bd16e3 (127.12.138.195:41525)
I20260811 02:56:09.064998 2169 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.12.138.195:43825
I20260811 02:56:09.065690 12843 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 02:56:09.072126 2214 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:56:09.067977 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46888 (local address 127.12.138.254:36093)
0811 02:56:09.068101 (+ 124us) server_negotiation.cc:206] Beginning negotiation
0811 02:56:09.068104 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 02:56:09.068291 (+ 187us) server_negotiation.cc:407] Connection header received
0811 02:56:09.068380 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:56:09.068382 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:56:09.068416 (+ 34us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:56:09.068483 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:56:09.068807 (+ 324us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:56:09.069360 (+ 553us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:56:09.070283 (+ 923us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:56:09.070483 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:56:09.070557 (+ 74us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:56:09.071497 (+ 940us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:56:09.071500 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:56:09.071502 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 02:56:09.071514 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:56:09.071550 (+ 36us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 02:56:09.071559 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:56:09.071719 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0811 02:56:09.071928 (+ 209us) server_negotiation.cc:299] Negotiation successful
0811 02:56:09.071960 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260811 02:56:09.075403 2166 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:46888:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 02:56:09.078641 2166 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 02:56:09.093227 2697 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:56:09.088763 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56862 (local address 127.12.138.193:45423)
0811 02:56:09.089110 (+ 347us) server_negotiation.cc:206] Beginning negotiation
0811 02:56:09.089120 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 02:56:09.089168 (+ 48us) server_negotiation.cc:407] Connection header received
0811 02:56:09.089245 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:56:09.089251 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:56:09.089424 (+ 173us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:56:09.089568 (+ 144us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 02:56:09.089757 (+ 189us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:56:09.090551 (+ 794us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:56:09.091904 (+ 1353us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:56:09.092511 (+ 607us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:56:09.092650 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:56:09.092836 (+ 186us) server_negotiation.cc:1035] Waiting for connection context
0811 02:56:09.092928 (+ 92us) server_negotiation.cc:299] Negotiation successful
0811 02:56:09.093026 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":249,"thread_start_us":138,"threads_started":1}
I20260811 02:56:09.093220 2695 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:56:09.087957 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.138.193:45423 (local address 127.0.0.1:56862)
0811 02:56:09.088201 (+ 244us) negotiation.cc:107] Waiting for socket to connect
0811 02:56:09.088216 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 02:56:09.088310 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:56:09.089547 (+ 1237us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:56:09.089550 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:56:09.089567 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 02:56:09.089651 (+ 84us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:56:09.089659 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:56:09.090680 (+ 1021us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:56:09.090684 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:56:09.091763 (+ 1079us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:56:09.091773 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:56:09.092644 (+ 871us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:56:09.092648 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:56:09.092751 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:56:09.092787 (+ 36us) client_negotiation.cc:769] Sending connection context
0811 02:56:09.092902 (+ 115us) client_negotiation.cc:240] Negotiation successful
0811 02:56:09.093027 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":178,"mutex_wait_us":40,"thread_start_us":108,"threads_started":1}
I20260811 02:56:09.093672 2690 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:56:09.087477 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.138.195:41525 (local address 127.0.0.1:43472)
0811 02:56:09.088177 (+ 700us) negotiation.cc:107] Waiting for socket to connect
0811 02:56:09.088191 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 02:56:09.088265 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:56:09.089337 (+ 1072us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:56:09.089341 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:56:09.089360 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 02:56:09.089461 (+ 101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:56:09.089468 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:56:09.090793 (+ 1325us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:56:09.090797 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:56:09.091949 (+ 1152us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:56:09.091960 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:56:09.093222 (+ 1262us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:56:09.093226 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:56:09.093367 (+ 141us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:56:09.093402 (+ 35us) client_negotiation.cc:769] Sending connection context
0811 02:56:09.093472 (+ 70us) client_negotiation.cc:240] Negotiation successful
0811 02:56:09.093532 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":630,"mutex_wait_us":26,"thread_start_us":105,"threads_started":1}
I20260811 02:56:09.094097 2696 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:56:09.088530 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43472 (local address 127.12.138.195:41525)
0811 02:56:09.088840 (+ 310us) server_negotiation.cc:206] Beginning negotiation
0811 02:56:09.088851 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 02:56:09.088877 (+ 26us) server_negotiation.cc:407] Connection header received
0811 02:56:09.088962 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:56:09.088969 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:56:09.089199 (+ 230us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:56:09.089360 (+ 161us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 02:56:09.089599 (+ 239us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:56:09.090680 (+ 1081us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:56:09.092078 (+ 1398us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:56:09.092842 (+ 764us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:56:09.092969 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:56:09.093130 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0811 02:56:09.093880 (+ 750us) server_negotiation.cc:299] Negotiation successful
0811 02:56:09.093978 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":218,"thread_start_us":126,"threads_started":1}
I20260811 02:56:09.094480 2694 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:56:09.087281 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.138.194:46571 (local address 127.0.0.1:42724)
0811 02:56:09.087823 (+ 542us) negotiation.cc:107] Waiting for socket to connect
0811 02:56:09.087874 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0811 02:56:09.088004 (+ 130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:56:09.088778 (+ 774us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:56:09.088784 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:56:09.088808 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 02:56:09.088968 (+ 160us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:56:09.088976 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:56:09.090452 (+ 1476us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:56:09.090456 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:56:09.091883 (+ 1427us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:56:09.091895 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:56:09.093226 (+ 1331us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:56:09.093229 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:56:09.094198 (+ 969us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:56:09.094222 (+ 24us) client_negotiation.cc:769] Sending connection context
0811 02:56:09.094301 (+ 79us) client_negotiation.cc:240] Negotiation successful
0811 02:56:09.094364 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":369,"mutex_wait_us":168,"thread_start_us":59,"threads_started":1}
I20260811 02:56:09.094813 2693 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:56:09.087428 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42724 (local address 127.12.138.194:46571)
0811 02:56:09.087933 (+ 505us) server_negotiation.cc:206] Beginning negotiation
0811 02:56:09.087946 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0811 02:56:09.087972 (+ 26us) server_negotiation.cc:407] Connection header received
0811 02:56:09.088234 (+ 262us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:56:09.088241 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:56:09.088447 (+ 206us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:56:09.088601 (+ 154us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 02:56:09.089106 (+ 505us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:56:09.090333 (+ 1227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:56:09.092042 (+ 1709us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:56:09.092842 (+ 800us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:56:09.092966 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:56:09.093130 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0811 02:56:09.094409 (+ 1279us) server_negotiation.cc:299] Negotiation successful
0811 02:56:09.094509 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":396,"thread_start_us":126,"threads_started":1}
I20260811 02:56:09.096596 2605 tablet_service.cc:1467] Processing CreateTablet for tablet b391997cd57d42788d0f909bd479a90e (DEFAULT_TABLE table=test-table [id=4b97d294512b469bb639e8ec5f7810df]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 02:56:09.096611 2305 tablet_service.cc:1467] Processing CreateTablet for tablet b391997cd57d42788d0f909bd479a90e (DEFAULT_TABLE table=test-table [id=4b97d294512b469bb639e8ec5f7810df]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 02:56:09.097625 2605 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b391997cd57d42788d0f909bd479a90e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 02:56:09.097729 2305 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b391997cd57d42788d0f909bd479a90e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 02:56:09.097573 2453 tablet_service.cc:1467] Processing CreateTablet for tablet b391997cd57d42788d0f909bd479a90e (DEFAULT_TABLE table=test-table [id=4b97d294512b469bb639e8ec5f7810df]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 02:56:09.098486 2453 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b391997cd57d42788d0f909bd479a90e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 02:56:09.103856 2698 tablet_bootstrap.cc:492] T b391997cd57d42788d0f909bd479a90e P 4f837d1fb2dc4e89b51025d6a8bd16e3: Bootstrap starting.
I20260811 02:56:09.105381 2699 tablet_bootstrap.cc:492] T b391997cd57d42788d0f909bd479a90e P 71b281515e0e416383d3224578671456: Bootstrap starting.
I20260811 02:56:09.105777 2700 tablet_bootstrap.cc:492] T b391997cd57d42788d0f909bd479a90e P 7274df7aa6204213b47ea681f9523e53: Bootstrap starting.
I20260811 02:56:09.106266 2698 tablet_bootstrap.cc:654] T b391997cd57d42788d0f909bd479a90e P 4f837d1fb2dc4e89b51025d6a8bd16e3: Neither blocks nor log segments found. Creating new log.
I20260811 02:56:09.107187 2698 log.cc:824] T b391997cd57d42788d0f909bd479a90e P 4f837d1fb2dc4e89b51025d6a8bd16e3: Log is configured to *not* fsync() on all Append() calls
I20260811 02:56:09.107496 2699 tablet_bootstrap.cc:654] T b391997cd57d42788d0f909bd479a90e P 71b281515e0e416383d3224578671456: Neither blocks nor log segments found. Creating new log.
I20260811 02:56:09.107599 2700 tablet_bootstrap.cc:654] T b391997cd57d42788d0f909bd479a90e P 7274df7aa6204213b47ea681f9523e53: Neither blocks nor log segments found. Creating new log.
I20260811 02:56:09.108367 2700 log.cc:824] T b391997cd57d42788d0f909bd479a90e P 7274df7aa6204213b47ea681f9523e53: Log is configured to *not* fsync() on all Append() calls
I20260811 02:56:09.108367 2699 log.cc:824] T b391997cd57d42788d0f909bd479a90e P 71b281515e0e416383d3224578671456: Log is configured to *not* fsync() on all Append() calls
I20260811 02:56:09.109297 2698 tablet_bootstrap.cc:492] T b391997cd57d42788d0f909bd479a90e P 4f837d1fb2dc4e89b51025d6a8bd16e3: No bootstrap required, opened a new log
I20260811 02:56:09.109503 2698 ts_tablet_manager.cc:1397] T b391997cd57d42788d0f909bd479a90e P 4f837d1fb2dc4e89b51025d6a8bd16e3: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260811 02:56:09.110333 2699 tablet_bootstrap.cc:492] T b391997cd57d42788d0f909bd479a90e P 71b281515e0e416383d3224578671456: No bootstrap required, opened a new log
I20260811 02:56:09.110334 2700 tablet_bootstrap.cc:492] T b391997cd57d42788d0f909bd479a90e P 7274df7aa6204213b47ea681f9523e53: No bootstrap required, opened a new log
I20260811 02:56:09.110514 2700 ts_tablet_manager.cc:1397] T b391997cd57d42788d0f909bd479a90e P 7274df7aa6204213b47ea681f9523e53: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260811 02:56:09.110513 2699 ts_tablet_manager.cc:1397] T b391997cd57d42788d0f909bd479a90e P 71b281515e0e416383d3224578671456: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260811 02:56:09.113431 2698 raft_consensus.cc:348] T b391997cd57d42788d0f909bd479a90e P 4f837d1fb2dc4e89b51025d6a8bd16e3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7274df7aa6204213b47ea681f9523e53" member_type: VOTER last_known_addr { host: "127.12.138.194" port: 46571 } } peers { permanent_uuid: "4f837d1fb2dc4e89b51025d6a8bd16e3" member_type: VOTER last_known_addr { host: "127.12.138.195" port: 41525 } } peers { permanent_uuid: "71b281515e0e416383d3224578671456" member_type: VOTER last_known_addr { host: "127.12.138.193" port: 45423 } }
I20260811 02:56:09.113686 2698 raft_consensus.cc:374] T b391997cd57d42788d0f909bd479a90e P 4f837d1fb2dc4e89b51025d6a8bd16e3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 02:56:09.113751 2698 raft_consensus.cc:729] T b391997cd57d42788d0f909bd479a90e P 4f837d1fb2dc4e89b51025d6a8bd16e3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4f837d1fb2dc4e89b51025d6a8bd16e3, State: Initialized, Role: FOLLOWER
I20260811 02:56:09.114164 2698 consensus_queue.cc:260] T b391997cd57d42788d0f909bd479a90e P 4f837d1fb2dc4e89b51025d6a8bd16e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7274df7aa6204213b47ea681f9523e53" member_type: VOTER last_known_addr { host: "127.12.138.194" port: 46571 } } peers { permanent_uuid: "4f837d1fb2dc4e89b51025d6a8bd16e3" member_type: VOTER last_known_addr { host: "127.12.138.195" port: 41525 } } peers { permanent_uuid: "71b281515e0e416383d3224578671456" member_type: VOTER last_known_addr { host: "127.12.138.193" port: 45423 } }
I20260811 02:56:09.114837 2679 heartbeater.cc:503] Master 127.12.138.254:36093 was elected leader, sending a full tablet report...
I20260811 02:56:09.114810 2699 raft_consensus.cc:348] T b391997cd57d42788d0f909bd479a90e P 71b281515e0e416383d3224578671456 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7274df7aa6204213b47ea681f9523e53" member_type: VOTER last_known_addr { host: "127.12.138.194" port: 46571 } } peers { permanent_uuid: "4f837d1fb2dc4e89b51025d6a8bd16e3" member_type: VOTER last_known_addr { host: "127.12.138.195" port: 41525 } } peers { permanent_uuid: "71b281515e0e416383d3224578671456" member_type: VOTER last_known_addr { host: "127.12.138.193" port: 45423 } }
I20260811 02:56:09.114810 2700 raft_consensus.cc:348] T b391997cd57d42788d0f909bd479a90e P 7274df7aa6204213b47ea681f9523e53 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7274df7aa6204213b47ea681f9523e53" member_type: VOTER last_known_addr { host: "127.12.138.194" port: 46571 } } peers { permanent_uuid: "4f837d1fb2dc4e89b51025d6a8bd16e3" member_type: VOTER last_known_addr { host: "127.12.138.195" port: 41525 } } peers { permanent_uuid: "71b281515e0e416383d3224578671456" member_type: VOTER last_known_addr { host: "127.12.138.193" port: 45423 } }
I20260811 02:56:09.115074 2699 raft_consensus.cc:374] T b391997cd57d42788d0f909bd479a90e P 71b281515e0e416383d3224578671456 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 02:56:09.115074 2700 raft_consensus.cc:374] T b391997cd57d42788d0f909bd479a90e P 7274df7aa6204213b47ea681f9523e53 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 02:56:09.115144 2699 raft_consensus.cc:729] T b391997cd57d42788d0f909bd479a90e P 71b281515e0e416383d3224578671456 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 71b281515e0e416383d3224578671456, State: Initialized, Role: FOLLOWER
I20260811 02:56:09.115154 2700 raft_consensus.cc:729] T b391997cd57d42788d0f909bd479a90e P 7274df7aa6204213b47ea681f9523e53 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7274df7aa6204213b47ea681f9523e53, State: Initialized, Role: FOLLOWER
I20260811 02:56:09.115605 2699 consensus_queue.cc:260] T b391997cd57d42788d0f909bd479a90e P 71b281515e0e416383d3224578671456 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7274df7aa6204213b47ea681f9523e53" member_type: VOTER last_known_addr { host: "127.12.138.194" port: 46571 } } peers { permanent_uuid: "4f837d1fb2dc4e89b51025d6a8bd16e3" member_type: VOTER last_known_addr { host: "127.12.138.195" port: 41525 } } peers { permanent_uuid: "71b281515e0e416383d3224578671456" member_type: VOTER last_known_addr { host: "127.12.138.193" port: 45423 } }
I20260811 02:56:09.115605 2700 consensus_queue.cc:260] T b391997cd57d42788d0f909bd479a90e P 7274df7aa6204213b47ea681f9523e53 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7274df7aa6204213b47ea681f9523e53" member_type: VOTER last_known_addr { host: "127.12.138.194" port: 46571 } } peers { permanent_uuid: "4f837d1fb2dc4e89b51025d6a8bd16e3" member_type: VOTER last_known_addr { host: "127.12.138.195" port: 41525 } } peers { permanent_uuid: "71b281515e0e416383d3224578671456" member_type: VOTER last_known_addr { host: "127.12.138.193" port: 45423 } }
I20260811 02:56:09.116642 2699 ts_tablet_manager.cc:1428] T b391997cd57d42788d0f909bd479a90e P 71b281515e0e416383d3224578671456: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260811 02:56:09.116652 2700 ts_tablet_manager.cc:1428] T b391997cd57d42788d0f909bd479a90e P 7274df7aa6204213b47ea681f9523e53: Time spent starting tablet: real 0.006s user 0.002s sys 0.005s
I20260811 02:56:09.117028 2530 heartbeater.cc:503] Master 127.12.138.254:36093 was elected leader, sending a full tablet report...
I20260811 02:56:09.117296 2379 heartbeater.cc:503] Master 127.12.138.254:36093 was elected leader, sending a full tablet report...
I20260811 02:56:09.118785 2698 ts_tablet_manager.cc:1428] T b391997cd57d42788d0f909bd479a90e P 4f837d1fb2dc4e89b51025d6a8bd16e3: Time spent starting tablet: real 0.009s user 0.006s sys 0.000s
W20260811 02:56:09.164738 2380 tablet.cc:2369] T b391997cd57d42788d0f909bd479a90e P 71b281515e0e416383d3224578671456: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 02:56:09.211871 2704 raft_consensus.cc:482] T b391997cd57d42788d0f909bd479a90e P 4f837d1fb2dc4e89b51025d6a8bd16e3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 02:56:09.212126 2704 raft_consensus.cc:504] T b391997cd57d42788d0f909bd479a90e P 4f837d1fb2dc4e89b51025d6a8bd16e3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7274df7aa6204213b47ea681f9523e53" member_type: VOTER last_known_addr { host: "127.12.138.194" port: 46571 } } peers { permanent_uuid: "4f837d1fb2dc4e89b51025d6a8bd16e3" member_type: VOTER last_known_addr { host: "127.12.138.195" port: 41525 } } peers { permanent_uuid: "71b281515e0e416383d3224578671456" member_type: VOTER last_known_addr { host: "127.12.138.193" port: 45423 } }
I20260811 02:56:09.213490 2704 leader_election.cc:288] T b391997cd57d42788d0f909bd479a90e P 4f837d1fb2dc4e89b51025d6a8bd16e3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7274df7aa6204213b47ea681f9523e53 (127.12.138.194:46571), 71b281515e0e416383d3224578671456 (127.12.138.193:45423)
I20260811 02:56:09.217408 2693 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:56:09.213804 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.138.195:54755 (local address 127.12.138.194:46571)
0811 02:56:09.213932 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0811 02:56:09.213939 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 02:56:09.213957 (+ 18us) server_negotiation.cc:407] Connection header received
0811 02:56:09.214136 (+ 179us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:56:09.214140 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:56:09.214194 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:56:09.214286 (+ 92us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 02:56:09.214600 (+ 314us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:56:09.215117 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:56:09.216213 (+ 1096us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:56:09.216770 (+ 557us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:56:09.216892 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:56:09.216942 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0811 02:56:09.217174 (+ 232us) server_negotiation.cc:299] Negotiation successful
0811 02:56:09.217272 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260811 02:56:09.217428 2681 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:56:09.213685 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.138.194:46571 (local address 127.12.138.195:54755)
0811 02:56:09.213886 (+ 201us) negotiation.cc:107] Waiting for socket to connect
0811 02:56:09.213901 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 02:56:09.213996 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:56:09.214314 (+ 318us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:56:09.214317 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:56:09.214334 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 02:56:09.214475 (+ 141us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:56:09.214481 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:56:09.215238 (+ 757us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:56:09.215243 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:56:09.216082 (+ 839us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:56:09.216091 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:56:09.216906 (+ 815us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:56:09.216909 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:56:09.216997 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:56:09.217020 (+ 23us) client_negotiation.cc:769] Sending connection context
0811 02:56:09.217099 (+ 79us) client_negotiation.cc:240] Negotiation successful
0811 02:56:09.217242 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":117,"mutex_wait_us":38,"spinlock_wait_cycles":9472}
I20260811 02:56:09.218010 2697 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:56:09.213937 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.138.195:53173 (local address 127.12.138.193:45423)
0811 02:56:09.214099 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0811 02:56:09.214105 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 02:56:09.214322 (+ 217us) server_negotiation.cc:407] Connection header received
0811 02:56:09.214482 (+ 160us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:56:09.214485 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:56:09.214538 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:56:09.214635 (+ 97us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 02:56:09.214897 (+ 262us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:56:09.215438 (+ 541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:56:09.216510 (+ 1072us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:56:09.217103 (+ 593us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:56:09.217299 (+ 196us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:56:09.217346 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0811 02:56:09.217763 (+ 417us) server_negotiation.cc:299] Negotiation successful
0811 02:56:09.217852 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260811 02:56:09.218339 2477 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b391997cd57d42788d0f909bd479a90e" candidate_uuid: "4f837d1fb2dc4e89b51025d6a8bd16e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7274df7aa6204213b47ea681f9523e53" is_pre_election: true
I20260811 02:56:09.218444 2707 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:56:09.213854 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.138.193:45423 (local address 127.12.138.195:53173)
0811 02:56:09.214221 (+ 367us) negotiation.cc:107] Waiting for socket to connect
0811 02:56:09.214234 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 02:56:09.214364 (+ 130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:56:09.214641 (+ 277us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:56:09.214644 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:56:09.214659 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 02:56:09.214783 (+ 124us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:56:09.214788 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:56:09.215546 (+ 758us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:56:09.215552 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:56:09.216350 (+ 798us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:56:09.216378 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:56:09.217285 (+ 907us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:56:09.217289 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:56:09.217395 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:56:09.217420 (+ 25us) client_negotiation.cc:769] Sending connection context
0811 02:56:09.218177 (+ 757us) client_negotiation.cc:240] Negotiation successful
0811 02:56:09.218283 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":307,"mutex_wait_us":2,"thread_start_us":143,"threads_started":1}
I20260811 02:56:09.218730 2477 raft_consensus.cc:2454] T b391997cd57d42788d0f909bd479a90e P 7274df7aa6204213b47ea681f9523e53 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f837d1fb2dc4e89b51025d6a8bd16e3 in term 0.
I20260811 02:56:09.219122 2329 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b391997cd57d42788d0f909bd479a90e" candidate_uuid: "4f837d1fb2dc4e89b51025d6a8bd16e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "71b281515e0e416383d3224578671456" is_pre_election: true
I20260811 02:56:09.219266 2556 leader_election.cc:302] T b391997cd57d42788d0f909bd479a90e P 4f837d1fb2dc4e89b51025d6a8bd16e3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f837d1fb2dc4e89b51025d6a8bd16e3, 7274df7aa6204213b47ea681f9523e53; no voters:
I20260811 02:56:09.219463 2329 raft_consensus.cc:2454] T b391997cd57d42788d0f909bd479a90e P 71b281515e0e416383d3224578671456 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f837d1fb2dc4e89b51025d6a8bd16e3 in term 0.
I20260811 02:56:09.219583 2704 raft_consensus.cc:2781] T b391997cd57d42788d0f909bd479a90e P 4f837d1fb2dc4e89b51025d6a8bd16e3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 02:56:09.219674 2704 raft_consensus.cc:482] T b391997cd57d42788d0f909bd479a90e P 4f837d1fb2dc4e89b51025d6a8bd16e3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 02:56:09.219733 2704 raft_consensus.cc:3037] T b391997cd57d42788d0f909bd479a90e P 4f837d1fb2dc4e89b51025d6a8bd16e3 [term 0 FOLLOWER]: Advancing to term 1
I20260811 02:56:09.220861 2704 raft_consensus.cc:504] T b391997cd57d42788d0f909bd479a90e P 4f837d1fb2dc4e89b51025d6a8bd16e3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7274df7aa6204213b47ea681f9523e53" member_type: VOTER last_known_addr { host: "127.12.138.194" port: 46571 } } peers { permanent_uuid: "4f837d1fb2dc4e89b51025d6a8bd16e3" member_type: VOTER last_known_addr { host: "127.12.138.195" port: 41525 } } peers { permanent_uuid: "71b281515e0e416383d3224578671456" member_type: VOTER last_known_addr { host: "127.12.138.193" port: 45423 } }
I20260811 02:56:09.221288 2704 leader_election.cc:288] T b391997cd57d42788d0f909bd479a90e P 4f837d1fb2dc4e89b51025d6a8bd16e3 [CANDIDATE]: Term 1 election: Requested vote from peers 7274df7aa6204213b47ea681f9523e53 (127.12.138.194:46571), 71b281515e0e416383d3224578671456 (127.12.138.193:45423)
I20260811 02:56:09.221601 2477 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b391997cd57d42788d0f909bd479a90e" candidate_uuid: "4f837d1fb2dc4e89b51025d6a8bd16e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7274df7aa6204213b47ea681f9523e53"
I20260811 02:56:09.221728 2477 raft_consensus.cc:3037] T b391997cd57d42788d0f909bd479a90e P 7274df7aa6204213b47ea681f9523e53 [term 0 FOLLOWER]: Advancing to term 1
I20260811 02:56:09.221706 2329 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b391997cd57d42788d0f909bd479a90e" candidate_uuid: "4f837d1fb2dc4e89b51025d6a8bd16e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "71b281515e0e416383d3224578671456"
I20260811 02:56:09.221830 2329 raft_consensus.cc:3037] T b391997cd57d42788d0f909bd479a90e P 71b281515e0e416383d3224578671456 [term 0 FOLLOWER]: Advancing to term 1
I20260811 02:56:09.222683 2477 raft_consensus.cc:2454] T b391997cd57d42788d0f909bd479a90e P 7274df7aa6204213b47ea681f9523e53 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f837d1fb2dc4e89b51025d6a8bd16e3 in term 1.
I20260811 02:56:09.222987 2556 leader_election.cc:302] T b391997cd57d42788d0f909bd479a90e P 4f837d1fb2dc4e89b51025d6a8bd16e3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f837d1fb2dc4e89b51025d6a8bd16e3, 7274df7aa6204213b47ea681f9523e53; no voters:
I20260811 02:56:09.223167 2704 raft_consensus.cc:2781] T b391997cd57d42788d0f909bd479a90e P 4f837d1fb2dc4e89b51025d6a8bd16e3 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 02:56:09.223201 2329 raft_consensus.cc:2454] T b391997cd57d42788d0f909bd479a90e P 71b281515e0e416383d3224578671456 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f837d1fb2dc4e89b51025d6a8bd16e3 in term 1.
I20260811 02:56:09.223433 2704 raft_consensus.cc:686] T b391997cd57d42788d0f909bd479a90e P 4f837d1fb2dc4e89b51025d6a8bd16e3 [term 1 LEADER]: Becoming Leader. State: Replica: 4f837d1fb2dc4e89b51025d6a8bd16e3, State: Running, Role: LEADER
I20260811 02:56:09.223807 2704 consensus_queue.cc:237] T b391997cd57d42788d0f909bd479a90e P 4f837d1fb2dc4e89b51025d6a8bd16e3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7274df7aa6204213b47ea681f9523e53" member_type: VOTER last_known_addr { host: "127.12.138.194" port: 46571 } } peers { permanent_uuid: "4f837d1fb2dc4e89b51025d6a8bd16e3" member_type: VOTER last_known_addr { host: "127.12.138.195" port: 41525 } } peers { permanent_uuid: "71b281515e0e416383d3224578671456" member_type: VOTER last_known_addr { host: "127.12.138.193" port: 45423 } }
I20260811 02:56:09.227373 2169 catalog_manager.cc:5495] T b391997cd57d42788d0f909bd479a90e P 4f837d1fb2dc4e89b51025d6a8bd16e3 reported cstate change: term changed from 0 to 1, leader changed from <none> to 4f837d1fb2dc4e89b51025d6a8bd16e3 (127.12.138.195). New cstate: current_term: 1 leader_uuid: "4f837d1fb2dc4e89b51025d6a8bd16e3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7274df7aa6204213b47ea681f9523e53" member_type: VOTER last_known_addr { host: "127.12.138.194" port: 46571 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4f837d1fb2dc4e89b51025d6a8bd16e3" member_type: VOTER last_known_addr { host: "127.12.138.195" port: 41525 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "71b281515e0e416383d3224578671456" member_type: VOTER last_known_addr { host: "127.12.138.193" port: 45423 } health_report { overall_health: UNKNOWN } } }
I20260811 02:56:09.251609 2696 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:56:09.247833 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43484 (local address 127.12.138.195:41525)
0811 02:56:09.247974 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0811 02:56:09.247978 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 02:56:09.247992 (+ 14us) server_negotiation.cc:407] Connection header received
0811 02:56:09.248073 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:56:09.248076 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:56:09.248136 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:56:09.248247 (+ 111us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 02:56:09.248517 (+ 270us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:56:09.249201 (+ 684us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:56:09.250172 (+ 971us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:56:09.250389 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:56:09.250471 (+ 82us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:56:09.250805 (+ 334us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 02:56:09.251216 (+ 411us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 02:56:09.251336 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0811 02:56:09.251459 (+ 123us) server_negotiation.cc:299] Negotiation successful
0811 02:56:09.251494 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260811 02:56:09.257637 2329 raft_consensus.cc:1261] T b391997cd57d42788d0f909bd479a90e P 71b281515e0e416383d3224578671456 [term 1 FOLLOWER]: Refusing update from remote peer 4f837d1fb2dc4e89b51025d6a8bd16e3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 02:56:09.257637 2477 raft_consensus.cc:1261] T b391997cd57d42788d0f909bd479a90e P 7274df7aa6204213b47ea681f9523e53 [term 1 FOLLOWER]: Refusing update from remote peer 4f837d1fb2dc4e89b51025d6a8bd16e3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 02:56:09.258402 2708 consensus_queue.cc:1035] T b391997cd57d42788d0f909bd479a90e P 4f837d1fb2dc4e89b51025d6a8bd16e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "71b281515e0e416383d3224578671456" member_type: VOTER last_known_addr { host: "127.12.138.193" port: 45423 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 02:56:09.258595 2704 consensus_queue.cc:1035] T b391997cd57d42788d0f909bd479a90e P 4f837d1fb2dc4e89b51025d6a8bd16e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7274df7aa6204213b47ea681f9523e53" member_type: VOTER last_known_addr { host: "127.12.138.194" port: 46571 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 02:56:09.266465 2714 mvcc.cc:205] Tried to move back new op lower bound from 7317163906071830528 to 7317163905943621632. Current Snapshot: MvccSnapshot[applied={T|T < 7317163906071830528}]
I20260811 02:56:09.275648 2168 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:46888:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 02:56:09.275837 2168 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:46888:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 02:56:09.278534 2168 catalog_manager.cc:5782] T 00000000000000000000000000000000 P a9f86b8a3ab6415b8e77a08de084718e: Sending DeleteTablet for 3 replicas of tablet b391997cd57d42788d0f909bd479a90e
I20260811 02:56:09.279460 2453 tablet_service.cc:1514] Processing DeleteTablet for tablet b391997cd57d42788d0f909bd479a90e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 02:56:09 UTC) from {username='slave'} at 127.0.0.1:42724
I20260811 02:56:09.279460 2605 tablet_service.cc:1514] Processing DeleteTablet for tablet b391997cd57d42788d0f909bd479a90e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 02:56:09 UTC) from {username='slave'} at 127.0.0.1:43472
I20260811 02:56:09.280026 2724 tablet_replica.cc:330] T b391997cd57d42788d0f909bd479a90e P 4f837d1fb2dc4e89b51025d6a8bd16e3: stopping tablet replica
I20260811 02:56:09.280016 2305 tablet_service.cc:1514] Processing DeleteTablet for tablet b391997cd57d42788d0f909bd479a90e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 02:56:09 UTC) from {username='slave'} at 127.0.0.1:56862
I20260811 02:56:09.280390 2724 raft_consensus.cc:2229] T b391997cd57d42788d0f909bd479a90e P 4f837d1fb2dc4e89b51025d6a8bd16e3 [term 1 LEADER]: Raft consensus shutting down.
I20260811 02:56:09.280587 2725 tablet_replica.cc:330] T b391997cd57d42788d0f909bd479a90e P 71b281515e0e416383d3224578671456: stopping tablet replica
I20260811 02:56:09.280840 2724 raft_consensus.cc:2258] T b391997cd57d42788d0f909bd479a90e P 4f837d1fb2dc4e89b51025d6a8bd16e3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 02:56:09.280998 2725 raft_consensus.cc:2229] T b391997cd57d42788d0f909bd479a90e P 71b281515e0e416383d3224578671456 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 02:56:09.281183 12843 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDawEwS/build/debug/bin/kudu with pid 2237
I20260811 02:56:09.282651 2723 tablet_replica.cc:330] T b391997cd57d42788d0f909bd479a90e P 7274df7aa6204213b47ea681f9523e53: stopping tablet replica
I20260811 02:56:09.282964 2723 raft_consensus.cc:2229] T b391997cd57d42788d0f909bd479a90e P 7274df7aa6204213b47ea681f9523e53 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 02:56:09.283311 2723 raft_consensus.cc:2258] T b391997cd57d42788d0f909bd479a90e P 7274df7aa6204213b47ea681f9523e53 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 02:56:09.285264 2724 ts_tablet_manager.cc:1905] T b391997cd57d42788d0f909bd479a90e P 4f837d1fb2dc4e89b51025d6a8bd16e3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 02:56:09.285547 2723 ts_tablet_manager.cc:1905] T b391997cd57d42788d0f909bd479a90e P 7274df7aa6204213b47ea681f9523e53: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 02:56:09.288450 2724 ts_tablet_manager.cc:1918] T b391997cd57d42788d0f909bd479a90e P 4f837d1fb2dc4e89b51025d6a8bd16e3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 02:56:09.288551 2724 log.cc:1196] T b391997cd57d42788d0f909bd479a90e P 4f837d1fb2dc4e89b51025d6a8bd16e3: Deleting WAL directory at /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786416707705494-12843-0/minicluster-data/ts-2/wal/wals/b391997cd57d42788d0f909bd479a90e
I20260811 02:56:09.288870 2724 ts_tablet_manager.cc:1939] T b391997cd57d42788d0f909bd479a90e P 4f837d1fb2dc4e89b51025d6a8bd16e3: Deleting consensus metadata
I20260811 02:56:09.290913 2723 ts_tablet_manager.cc:1918] T b391997cd57d42788d0f909bd479a90e P 7274df7aa6204213b47ea681f9523e53: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 02:56:09.291040 2723 log.cc:1196] T b391997cd57d42788d0f909bd479a90e P 7274df7aa6204213b47ea681f9523e53: Deleting WAL directory at /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786416707705494-12843-0/minicluster-data/ts-1/wal/wals/b391997cd57d42788d0f909bd479a90e
I20260811 02:56:09.291400 2723 ts_tablet_manager.cc:1939] T b391997cd57d42788d0f909bd479a90e P 7274df7aa6204213b47ea681f9523e53: Deleting consensus metadata
I20260811 02:56:09.292346 2151 catalog_manager.cc:4841] TS 4f837d1fb2dc4e89b51025d6a8bd16e3 (127.12.138.195:41525): tablet b391997cd57d42788d0f909bd479a90e (table test-table [id=4b97d294512b469bb639e8ec5f7810df]) successfully deleted
I20260811 02:56:09.292718 2154 catalog_manager.cc:4841] TS 7274df7aa6204213b47ea681f9523e53 (127.12.138.194:46571): tablet b391997cd57d42788d0f909bd479a90e (table test-table [id=4b97d294512b469bb639e8ec5f7810df]) successfully deleted
W20260811 02:56:09.298583 2151 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.12.138.193:45423 (error 108)
I20260811 02:56:09.298861 12843 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDawEwS/build/debug/bin/kudu with pid 2383
I20260811 02:56:09.307813 2694 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:56:09.307411 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.138.193:45423 (local address 127.0.0.1:56874)
0811 02:56:09.307625 (+ 214us) negotiation.cc:107] Waiting for socket to connect
0811 02:56:09.307710 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.138.193:45423: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":128}
W20260811 02:56:09.308032 2151 catalog_manager.cc:4584] TS 71b281515e0e416383d3224578671456 (127.12.138.193:45423): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet b391997cd57d42788d0f909bd479a90e: Network error: Client connection negotiation failed: client connection to 127.12.138.193:45423: connect: Connection refused (error 111)
I20260811 02:56:09.313992 12843 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDawEwS/build/debug/bin/kudu with pid 2533
I20260811 02:56:09.324380 12843 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDawEwS/build/debug/bin/kudu with pid 2135
2026-08-11T02:56:09Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/3 (1152 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/4
Loading random data
Initializing database '/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786416707705494-12843-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 02:56:09 dist-test-slave-9gcw krb5kdc[2730](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 02:56:09 dist-test-slave-9gcw krb5kdc[2730](info): set up 2 sockets
Aug 11 02:56:09 dist-test-slave-9gcw krb5kdc[2730](info): commencing operation
krb5kdc: starting...
W20260811 02:56:10.476531 12843 mini_kdc.cc:121] Time spent starting KDC: real 1.127s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786416707705494-12843-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786416707705494-12843-0/krb5kdc/test-user.keytab.
Aug 11 02:56:10 dist-test-slave-9gcw krb5kdc[2730](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416970, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T02:56:10Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T02:56:10Z Disabled control of system clock
WARNING: no policy specified for kudu/127.12.138.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.138.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.138.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786416707705494-12843-0/krb5kdc/kudu_127.12.138.254.keytab.
Entry for principal kudu/127.12.138.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786416707705494-12843-0/krb5kdc/kudu_127.12.138.254.keytab.
I20260811 02:56:10.612600 12843 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDawEwS/build/debug/bin/kudu
/tmp/dist-test-taskDawEwS/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786416707705494-12843-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786416707705494-12843-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786416707705494-12843-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786416707705494-12843-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.138.254:35843
--webserver_interface=127.12.138.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786416707705494-12843-0/krb5kdc/kudu_127.12.138.254.keytab
--principal=kudu/127.12.138.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.138.212:34411
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.12.138.254:35843
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786416707705494-12843-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786416707705494-12843-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786416707705494-12843-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 02:56:10.728780 2745 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 02:56:10.729103 2745 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 02:56:10.729209 2745 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 02:56:10.733316 2745 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 02:56:10.733428 2745 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 02:56:10.733467 2745 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 02:56:10.733502 2745 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 02:56:10.733541 2745 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 02:56:10.739013 2745 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.138.212:34411
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786416707705494-12843-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786416707705494-12843-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.12.138.254:35843
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786416707705494-12843-0/krb5kdc/kudu_127.12.138.254.keytab
--principal=kudu/127.12.138.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.138.254:35843
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786416707705494-12843-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.12.138.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2745
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786416707705494-12843-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 02:09:49 UTC on ccycloud.quasar-fsnqqc.root.comops.site
build id 4579570
I20260811 02:56:10.740192 2745 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 02:56:10.741281 2745 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 02:56:10.750120 2750 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:56:10.750139 2754 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:56:10.750136 2752 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:56:10.750528 2745 server_base.cc:1034] running on GCE node
I20260811 02:56:10.751096 2745 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 02:56:10.752281 2745 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 02:56:10.753474 2745 hybrid_clock.cc:648] HybridClock initialized: now 1786416970753449 us; error 37 us; skew 500 ppm
Aug 11 02:56:10 dist-test-slave-9gcw krb5kdc[2730](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416970, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.138.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 02:56:10.759910 2745 init.cc:377] Logged in from keytab as kudu/127.12.138.254@KRBTEST.COM (short username kudu)
I20260811 02:56:10.760996 2745 webserver.cc:460] Webserver started at http://127.12.138.254:43125/ using document root <none> and password file <none>
I20260811 02:56:10.761669 2745 fs_manager.cc:359] Metadata directory not provided
I20260811 02:56:10.761740 2745 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 02:56:10.761976 2745 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 02:56:10.764119 2745 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786416707705494-12843-0/minicluster-data/master-0/data/instance:
uuid: "528aa09ab93a460dbdf08dccfd86354f"
format_stamp: "Formatted at 2026-08-11 02:56:10 on dist-test-slave-9gcw"
server_key: "f1c3c0ff08b236697efce75809a923cd"
server_key_iv: "4e662a5e9186283609ede9fd5ca3e457"
server_key_version: "encryptionkey@0"
I20260811 02:56:10.764710 2745 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786416707705494-12843-0/minicluster-data/master-0/wal/instance:
uuid: "528aa09ab93a460dbdf08dccfd86354f"
format_stamp: "Formatted at 2026-08-11 02:56:10 on dist-test-slave-9gcw"
server_key: "f1c3c0ff08b236697efce75809a923cd"
server_key_iv: "4e662a5e9186283609ede9fd5ca3e457"
server_key_version: "encryptionkey@0"
I20260811 02:56:10.768556 2745 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260811 02:56:10.771063 2761 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 02:56:10.772289 2745 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260811 02:56:10.772415 2745 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786416707705494-12843-0/minicluster-data/master-0/data,/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786416707705494-12843-0/minicluster-data/master-0/wal
uuid: "528aa09ab93a460dbdf08dccfd86354f"
format_stamp: "Formatted at 2026-08-11 02:56:10 on dist-test-slave-9gcw"
server_key: "f1c3c0ff08b236697efce75809a923cd"
server_key_iv: "4e662a5e9186283609ede9fd5ca3e457"
server_key_version: "encryptionkey@0"
I20260811 02:56:10.772526 2745 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786416707705494-12843-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786416707705494-12843-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786416707705494-12843-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 02:56:10.786118 2745 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 02:56:10.789822 2745 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 02:56:10.790094 2745 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 02:56:10.797749 2745 rpc_server.cc:307] RPC server started. Bound to: 127.12.138.254:35843
I20260811 02:56:10.797767 2818 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.138.254:35843 every 8 connection(s)
I20260811 02:56:10.798760 2745 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786416707705494-12843-0/minicluster-data/master-0/data/info.pb
I20260811 02:56:10.800386 12843 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDawEwS/build/debug/bin/kudu as pid 2745
I20260811 02:56:10.800540 12843 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786416707705494-12843-0/minicluster-data/master-0/wal/instance
I20260811 02:56:10.800911 12843 external_mini_cluster.cc:1428] Setting key dbe9ead522981c4354d6cd72238309e7
I20260811 02:56:10.802079 2819 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 02:56:10.809110 2819 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 528aa09ab93a460dbdf08dccfd86354f: Bootstrap starting.
Aug 11 02:56:10 dist-test-slave-9gcw krb5kdc[2730](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416970, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.12.138.254@KRBTEST.COM
I20260811 02:56:10.811745 2819 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 528aa09ab93a460dbdf08dccfd86354f: Neither blocks nor log segments found. Creating new log.
I20260811 02:56:10.812721 2819 log.cc:824] T 00000000000000000000000000000000 P 528aa09ab93a460dbdf08dccfd86354f: Log is configured to *not* fsync() on all Append() calls
I20260811 02:56:10.815057 2819 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 528aa09ab93a460dbdf08dccfd86354f: No bootstrap required, opened a new log
I20260811 02:56:10.816710 2822 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:56:10.802511 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45108 (local address 127.12.138.254:35843)
0811 02:56:10.802958 (+ 447us) server_negotiation.cc:206] Beginning negotiation
0811 02:56:10.802978 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0811 02:56:10.803021 (+ 43us) server_negotiation.cc:407] Connection header received
0811 02:56:10.803906 (+ 885us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:56:10.803929 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:56:10.804367 (+ 438us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:56:10.804752 (+ 385us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:56:10.805523 (+ 771us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:56:10.807091 (+ 1568us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:56:10.807974 (+ 883us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:56:10.808665 (+ 691us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:56:10.808831 (+ 166us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:56:10.811595 (+ 2764us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:56:10.811604 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:56:10.811624 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:56:10.811663 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:56:10.813597 (+ 1934us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:56:10.814025 (+ 428us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:56:10.814030 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:56:10.814036 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:56:10.814172 (+ 136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:56:10.814461 (+ 289us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:56:10.814464 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:56:10.814466 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:56:10.814969 (+ 503us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:56:10.815165 (+ 196us) server_negotiation.cc:1035] Waiting for connection context
0811 02:56:10.815479 (+ 314us) server_negotiation.cc:299] Negotiation successful
0811 02:56:10.815778 (+ 299us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":298,"thread_start_us":133,"threads_started":1}
I20260811 02:56:10.819433 2819 raft_consensus.cc:348] T 00000000000000000000000000000000 P 528aa09ab93a460dbdf08dccfd86354f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "528aa09ab93a460dbdf08dccfd86354f" member_type: VOTER last_known_addr { host: "127.12.138.254" port: 35843 } }
I20260811 02:56:10.819651 2819 raft_consensus.cc:374] T 00000000000000000000000000000000 P 528aa09ab93a460dbdf08dccfd86354f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 02:56:10.819694 2819 raft_consensus.cc:729] T 00000000000000000000000000000000 P 528aa09ab93a460dbdf08dccfd86354f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 528aa09ab93a460dbdf08dccfd86354f, State: Initialized, Role: FOLLOWER
I20260811 02:56:10.820053 2819 consensus_queue.cc:260] T 00000000000000000000000000000000 P 528aa09ab93a460dbdf08dccfd86354f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "528aa09ab93a460dbdf08dccfd86354f" member_type: VOTER last_known_addr { host: "127.12.138.254" port: 35843 } }
I20260811 02:56:10.820168 2819 raft_consensus.cc:388] T 00000000000000000000000000000000 P 528aa09ab93a460dbdf08dccfd86354f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 02:56:10.820216 2819 raft_consensus.cc:482] T 00000000000000000000000000000000 P 528aa09ab93a460dbdf08dccfd86354f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 02:56:10.820339 2819 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 528aa09ab93a460dbdf08dccfd86354f [term 0 FOLLOWER]: Advancing to term 1
I20260811 02:56:10.821446 2819 raft_consensus.cc:504] T 00000000000000000000000000000000 P 528aa09ab93a460dbdf08dccfd86354f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "528aa09ab93a460dbdf08dccfd86354f" member_type: VOTER last_known_addr { host: "127.12.138.254" port: 35843 } }
I20260811 02:56:10.821758 2819 leader_election.cc:302] T 00000000000000000000000000000000 P 528aa09ab93a460dbdf08dccfd86354f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 528aa09ab93a460dbdf08dccfd86354f; no voters:
I20260811 02:56:10.822046 2819 leader_election.cc:288] T 00000000000000000000000000000000 P 528aa09ab93a460dbdf08dccfd86354f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 02:56:10.822201 2824 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 528aa09ab93a460dbdf08dccfd86354f [term 1 FOLLOWER]: Leader election won for term 1
I20260811 02:56:10.822486 2824 raft_consensus.cc:686] T 00000000000000000000000000000000 P 528aa09ab93a460dbdf08dccfd86354f [term 1 LEADER]: Becoming Leader. State: Replica: 528aa09ab93a460dbdf08dccfd86354f, State: Running, Role: LEADER
I20260811 02:56:10.822817 2824 consensus_queue.cc:237] T 00000000000000000000000000000000 P 528aa09ab93a460dbdf08dccfd86354f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "528aa09ab93a460dbdf08dccfd86354f" member_type: VOTER last_known_addr { host: "127.12.138.254" port: 35843 } }
I20260811 02:56:10.823262 2819 sys_catalog.cc:564] T 00000000000000000000000000000000 P 528aa09ab93a460dbdf08dccfd86354f [sys.catalog]: configured and running, proceeding with master startup.
I20260811 02:56:10.824465 2826 sys_catalog.cc:455] T 00000000000000000000000000000000 P 528aa09ab93a460dbdf08dccfd86354f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 528aa09ab93a460dbdf08dccfd86354f. Latest consensus state: current_term: 1 leader_uuid: "528aa09ab93a460dbdf08dccfd86354f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "528aa09ab93a460dbdf08dccfd86354f" member_type: VOTER last_known_addr { host: "127.12.138.254" port: 35843 } } }
I20260811 02:56:10.824602 2826 sys_catalog.cc:458] T 00000000000000000000000000000000 P 528aa09ab93a460dbdf08dccfd86354f [sys.catalog]: This master's current role is: LEADER
I20260811 02:56:10.824793 2825 sys_catalog.cc:455] T 00000000000000000000000000000000 P 528aa09ab93a460dbdf08dccfd86354f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "528aa09ab93a460dbdf08dccfd86354f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "528aa09ab93a460dbdf08dccfd86354f" member_type: VOTER last_known_addr { host: "127.12.138.254" port: 35843 } } }
I20260811 02:56:10.824999 2825 sys_catalog.cc:458] T 00000000000000000000000000000000 P 528aa09ab93a460dbdf08dccfd86354f [sys.catalog]: This master's current role is: LEADER
I20260811 02:56:10.825803 2834 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 02:56:10.828905 2834 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 02:56:10.834489 2834 catalog_manager.cc:1347] Generated new cluster ID: 1ff50ac07ee94acb8bd33851f1a95915
I20260811 02:56:10.834573 2834 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 02:56:10.840106 2834 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 02:56:10.841018 2834 catalog_manager.cc:1504] Loading token signing keys...
I20260811 02:56:10.846869 2834 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 528aa09ab93a460dbdf08dccfd86354f: Generated new TSK 0
I20260811 02:56:10.847546 2834 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.12.138.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.138.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.138.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786416707705494-12843-0/krb5kdc/kudu_127.12.138.193.keytab.
Entry for principal kudu/127.12.138.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786416707705494-12843-0/krb5kdc/kudu_127.12.138.193.keytab.
I20260811 02:56:10.880867 12843 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDawEwS/build/debug/bin/kudu
/tmp/dist-test-taskDawEwS/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786416707705494-12843-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786416707705494-12843-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786416707705494-12843-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786416707705494-12843-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.138.193:0
--local_ip_for_outbound_sockets=127.12.138.193
--webserver_interface=127.12.138.193
--webserver_port=0
--tserver_master_addrs=127.12.138.254:35843
--keytab_file=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786416707705494-12843-0/krb5kdc/kudu_127.12.138.193.keytab
--principal=kudu/127.12.138.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.138.212:34411
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786416707705494-12843-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786416707705494-12843-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786416707705494-12843-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 02:56:11.010622 2849 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 02:56:11.010901 2849 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 02:56:11.010946 2849 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 02:56:11.015065 2849 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 02:56:11.015132 2849 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 02:56:11.015193 2849 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.138.193
I20260811 02:56:11.020713 2849 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.138.212:34411
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786416707705494-12843-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786416707705494-12843-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786416707705494-12843-0/krb5kdc/kudu_127.12.138.193.keytab
--principal=kudu/127.12.138.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.138.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786416707705494-12843-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.12.138.193
--webserver_port=0
--tserver_master_addrs=127.12.138.254:35843
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2849
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.138.193
--log_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786416707705494-12843-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 02:09:49 UTC on ccycloud.quasar-fsnqqc.root.comops.site
build id 4579570
I20260811 02:56:11.021842 2849 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 02:56:11.022876 2849 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 02:56:11.033844 2855 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:56:11.033737 2854 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:56:11.033735 2858 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:56:11.034358 2849 server_base.cc:1034] running on GCE node
I20260811 02:56:11.034809 2849 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 02:56:11.035527 2849 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 02:56:11.036737 2849 hybrid_clock.cc:648] HybridClock initialized: now 1786416971036700 us; error 58 us; skew 500 ppm
Aug 11 02:56:11 dist-test-slave-9gcw krb5kdc[2730](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416971, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.138.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 02:56:11.039896 2849 init.cc:377] Logged in from keytab as kudu/127.12.138.193@KRBTEST.COM (short username kudu)
I20260811 02:56:11.040890 2849 webserver.cc:460] Webserver started at http://127.12.138.193:39691/ using document root <none> and password file <none>
I20260811 02:56:11.041591 2849 fs_manager.cc:359] Metadata directory not provided
I20260811 02:56:11.041680 2849 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 02:56:11.041913 2849 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 02:56:11.044086 2849 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786416707705494-12843-0/minicluster-data/ts-0/data/instance:
uuid: "566fc2a50bfa404ebfa81c06ec9a85c9"
format_stamp: "Formatted at 2026-08-11 02:56:11 on dist-test-slave-9gcw"
server_key: "80857b6931e2230f853ddd7ffd868767"
server_key_iv: "0aa50113ca447ef9dffcaf57e91700f8"
server_key_version: "encryptionkey@0"
I20260811 02:56:11.044696 2849 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786416707705494-12843-0/minicluster-data/ts-0/wal/instance:
uuid: "566fc2a50bfa404ebfa81c06ec9a85c9"
format_stamp: "Formatted at 2026-08-11 02:56:11 on dist-test-slave-9gcw"
server_key: "80857b6931e2230f853ddd7ffd868767"
server_key_iv: "0aa50113ca447ef9dffcaf57e91700f8"
server_key_version: "encryptionkey@0"
I20260811 02:56:11.048784 2849 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.003s
I20260811 02:56:11.051638 2867 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 02:56:11.053073 2849 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260811 02:56:11.053274 2849 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786416707705494-12843-0/minicluster-data/ts-0/data,/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786416707705494-12843-0/minicluster-data/ts-0/wal
uuid: "566fc2a50bfa404ebfa81c06ec9a85c9"
format_stamp: "Formatted at 2026-08-11 02:56:11 on dist-test-slave-9gcw"
server_key: "80857b6931e2230f853ddd7ffd868767"
server_key_iv: "0aa50113ca447ef9dffcaf57e91700f8"
server_key_version: "encryptionkey@0"
I20260811 02:56:11.053433 2849 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786416707705494-12843-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786416707705494-12843-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786416707705494-12843-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 02:56:11.091243 2849 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 02:56:11.094739 2849 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 02:56:11.094964 2849 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 02:56:11.095631 2849 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 02:56:11.096561 2849 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 02:56:11.096643 2849 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 02:56:11.096740 2849 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 02:56:11.096791 2849 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 02:56:11.106918 2849 rpc_server.cc:307] RPC server started. Bound to: 127.12.138.193:34809
I20260811 02:56:11.106943 2987 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.138.193:34809 every 8 connection(s)
I20260811 02:56:11.107939 2849 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786416707705494-12843-0/minicluster-data/ts-0/data/info.pb
I20260811 02:56:11.108923 12843 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDawEwS/build/debug/bin/kudu as pid 2849
I20260811 02:56:11.109062 12843 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786416707705494-12843-0/minicluster-data/ts-0/wal/instance
I20260811 02:56:11.109438 12843 external_mini_cluster.cc:1428] Setting key aaaf51431bc80925af17f755d7acad4d
Aug 11 02:56:11 dist-test-slave-9gcw krb5kdc[2730](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416971, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.138.193@KRBTEST.COM for kudu/127.12.138.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.12.138.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.138.194@KRBTEST.COM" created.
I20260811 02:56:11.125442 2822 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:56:11.110114 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.138.193:46249 (local address 127.12.138.254:35843)
0811 02:56:11.110296 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0811 02:56:11.110306 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 02:56:11.111351 (+ 1045us) server_negotiation.cc:407] Connection header received
0811 02:56:11.112913 (+ 1562us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:56:11.112919 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:56:11.112985 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:56:11.113088 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:56:11.115013 (+ 1925us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:56:11.115971 (+ 958us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:56:11.117362 (+ 1391us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:56:11.117849 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:56:11.118016 (+ 167us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:56:11.120991 (+ 2975us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:56:11.120995 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:56:11.120998 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:56:11.121032 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:56:11.122667 (+ 1635us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:56:11.123299 (+ 632us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:56:11.123302 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:56:11.123304 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:56:11.123422 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:56:11.123927 (+ 505us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:56:11.123932 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:56:11.123933 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:56:11.124261 (+ 328us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:56:11.124373 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0811 02:56:11.124942 (+ 569us) server_negotiation.cc:299] Negotiation successful
0811 02:56:11.125108 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260811 02:56:11.126335 2991 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:56:11.110619 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.138.254:35843 (local address 127.12.138.193:46249)
0811 02:56:11.111181 (+ 562us) negotiation.cc:107] Waiting for socket to connect
0811 02:56:11.111241 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0811 02:56:11.112683 (+ 1442us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:56:11.113374 (+ 691us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:56:11.113385 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:56:11.113982 (+ 597us) client_negotiation.cc:189] Negotiated authn=SASL
0811 02:56:11.114808 (+ 826us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:56:11.114836 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:56:11.116138 (+ 1302us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:56:11.116141 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:56:11.117131 (+ 990us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:56:11.117217 (+ 86us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:56:11.118094 (+ 877us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:56:11.118097 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:56:11.118298 (+ 201us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:56:11.119070 (+ 772us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 02:56:11.119089 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0811 02:56:11.120771 (+ 1682us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 02:56:11.122885 (+ 2114us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:56:11.122891 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:56:11.122905 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:56:11.123193 (+ 288us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:56:11.123569 (+ 376us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:56:11.123572 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:56:11.123573 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:56:11.123790 (+ 217us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:56:11.124370 (+ 580us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 02:56:11.124376 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 02:56:11.124658 (+ 282us) client_negotiation.cc:769] Sending connection context
0811 02:56:11.124888 (+ 230us) client_negotiation.cc:240] Negotiation successful
0811 02:56:11.125319 (+ 431us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":339,"thread_start_us":172,"threads_started":1}
I20260811 02:56:11.127771 2989 heartbeater.cc:347] Connected to a master server at 127.12.138.254:35843
I20260811 02:56:11.128063 2989 heartbeater.cc:464] Registering TS with master...
I20260811 02:56:11.128791 2989 heartbeater.cc:511] Master 127.12.138.254:35843 requested a full tablet report, sending...
I20260811 02:56:11.130759 2780 ts_manager.cc:194] Registered new tserver with Master: 566fc2a50bfa404ebfa81c06ec9a85c9 (127.12.138.193:34809)
I20260811 02:56:11.132776 2780 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.12.138.193@KRBTEST.COM'} at 127.12.138.193:46249
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.138.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786416707705494-12843-0/krb5kdc/kudu_127.12.138.194.keytab.
Entry for principal kudu/127.12.138.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786416707705494-12843-0/krb5kdc/kudu_127.12.138.194.keytab.
I20260811 02:56:11.143934 12843 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDawEwS/build/debug/bin/kudu
/tmp/dist-test-taskDawEwS/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786416707705494-12843-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786416707705494-12843-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786416707705494-12843-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786416707705494-12843-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.138.194:0
--local_ip_for_outbound_sockets=127.12.138.194
--webserver_interface=127.12.138.194
--webserver_port=0
--tserver_master_addrs=127.12.138.254:35843
--keytab_file=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786416707705494-12843-0/krb5kdc/kudu_127.12.138.194.keytab
--principal=kudu/127.12.138.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.138.212:34411
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786416707705494-12843-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786416707705494-12843-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786416707705494-12843-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 02:56:11.261468 2994 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 02:56:11.261819 2994 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 02:56:11.261873 2994 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 02:56:11.265947 2994 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 02:56:11.266014 2994 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 02:56:11.266074 2994 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.138.194
I20260811 02:56:11.271636 2994 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.138.212:34411
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786416707705494-12843-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786416707705494-12843-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786416707705494-12843-0/krb5kdc/kudu_127.12.138.194.keytab
--principal=kudu/127.12.138.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.138.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786416707705494-12843-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.12.138.194
--webserver_port=0
--tserver_master_addrs=127.12.138.254:35843
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2994
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.138.194
--log_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786416707705494-12843-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 02:09:49 UTC on ccycloud.quasar-fsnqqc.root.comops.site
build id 4579570
I20260811 02:56:11.272750 2994 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 02:56:11.273918 2994 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 02:56:11.284606 3000 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:56:11.284749 2994 server_base.cc:1034] running on GCE node
W20260811 02:56:11.284598 2999 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:56:11.284596 3002 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:56:11.285452 2994 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 02:56:11.286161 2994 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 02:56:11.287353 2994 hybrid_clock.cc:648] HybridClock initialized: now 1786416971287335 us; error 33 us; skew 500 ppm
Aug 11 02:56:11 dist-test-slave-9gcw krb5kdc[2730](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416971, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.138.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 02:56:11.290340 2994 init.cc:377] Logged in from keytab as kudu/127.12.138.194@KRBTEST.COM (short username kudu)
I20260811 02:56:11.291329 2994 webserver.cc:460] Webserver started at http://127.12.138.194:36313/ using document root <none> and password file <none>
I20260811 02:56:11.291919 2994 fs_manager.cc:359] Metadata directory not provided
I20260811 02:56:11.292011 2994 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 02:56:11.292263 2994 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 02:56:11.294499 2994 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786416707705494-12843-0/minicluster-data/ts-1/data/instance:
uuid: "e2d485f769814847ba1861dcfb5c4318"
format_stamp: "Formatted at 2026-08-11 02:56:11 on dist-test-slave-9gcw"
server_key: "12cfe6c23e30155576c3a564de37c5cf"
server_key_iv: "cbb60417663ad74bb787adb3fc52d8ae"
server_key_version: "encryptionkey@0"
I20260811 02:56:11.295099 2994 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786416707705494-12843-0/minicluster-data/ts-1/wal/instance:
uuid: "e2d485f769814847ba1861dcfb5c4318"
format_stamp: "Formatted at 2026-08-11 02:56:11 on dist-test-slave-9gcw"
server_key: "12cfe6c23e30155576c3a564de37c5cf"
server_key_iv: "cbb60417663ad74bb787adb3fc52d8ae"
server_key_version: "encryptionkey@0"
I20260811 02:56:11.299476 2994 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260811 02:56:11.302196 3010 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 02:56:11.303483 2994 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 02:56:11.303629 2994 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786416707705494-12843-0/minicluster-data/ts-1/data,/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786416707705494-12843-0/minicluster-data/ts-1/wal
uuid: "e2d485f769814847ba1861dcfb5c4318"
format_stamp: "Formatted at 2026-08-11 02:56:11 on dist-test-slave-9gcw"
server_key: "12cfe6c23e30155576c3a564de37c5cf"
server_key_iv: "cbb60417663ad74bb787adb3fc52d8ae"
server_key_version: "encryptionkey@0"
I20260811 02:56:11.303757 2994 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786416707705494-12843-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786416707705494-12843-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786416707705494-12843-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 02:56:11.327562 2994 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 02:56:11.330989 2994 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 02:56:11.331210 2994 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 02:56:11.331903 2994 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 02:56:11.332785 2994 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 02:56:11.332855 2994 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 02:56:11.332923 2994 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 02:56:11.332969 2994 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 02:56:11.342500 2994 rpc_server.cc:307] RPC server started. Bound to: 127.12.138.194:43359
I20260811 02:56:11.342597 3137 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.138.194:43359 every 8 connection(s)
I20260811 02:56:11.343462 2994 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786416707705494-12843-0/minicluster-data/ts-1/data/info.pb
I20260811 02:56:11.351969 12843 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDawEwS/build/debug/bin/kudu as pid 2994
I20260811 02:56:11.352123 12843 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786416707705494-12843-0/minicluster-data/ts-1/wal/instance
I20260811 02:56:11.352450 12843 external_mini_cluster.cc:1428] Setting key 38e5cce8141a3f7f5ce98f4ef41defe5
Aug 11 02:56:11 dist-test-slave-9gcw krb5kdc[2730](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416971, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.138.194@KRBTEST.COM for kudu/127.12.138.254@KRBTEST.COM
I20260811 02:56:11.358409 2822 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:56:11.345356 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.138.194:43489 (local address 127.12.138.254:35843)
0811 02:56:11.345511 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0811 02:56:11.345516 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 02:56:11.346492 (+ 976us) server_negotiation.cc:407] Connection header received
0811 02:56:11.348025 (+ 1533us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:56:11.348027 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:56:11.348076 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:56:11.348155 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:56:11.349796 (+ 1641us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:56:11.350352 (+ 556us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:56:11.351554 (+ 1202us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:56:11.351788 (+ 234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:56:11.351900 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:56:11.354642 (+ 2742us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:56:11.354646 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:56:11.354648 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:56:11.354681 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:56:11.356125 (+ 1444us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:56:11.356677 (+ 552us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:56:11.356681 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:56:11.356683 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:56:11.356769 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:56:11.357168 (+ 399us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:56:11.357172 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:56:11.357173 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:56:11.357434 (+ 261us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:56:11.357543 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0811 02:56:11.358072 (+ 529us) server_negotiation.cc:299] Negotiation successful
0811 02:56:11.358216 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260811 02:56:11.359405 3140 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:56:11.345794 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.138.254:35843 (local address 127.12.138.194:43489)
0811 02:56:11.346313 (+ 519us) negotiation.cc:107] Waiting for socket to connect
0811 02:56:11.346376 (+ 63us) client_negotiation.cc:174] Beginning negotiation
0811 02:56:11.347813 (+ 1437us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:56:11.348374 (+ 561us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:56:11.348381 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:56:11.348888 (+ 507us) client_negotiation.cc:189] Negotiated authn=SASL
0811 02:56:11.349640 (+ 752us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:56:11.349660 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:56:11.350487 (+ 827us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:56:11.350490 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:56:11.351416 (+ 926us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:56:11.351425 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:56:11.351942 (+ 517us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:56:11.351946 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:56:11.352172 (+ 226us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:56:11.352867 (+ 695us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 02:56:11.352885 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0811 02:56:11.354475 (+ 1590us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 02:56:11.356276 (+ 1801us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:56:11.356281 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:56:11.356297 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:56:11.356565 (+ 268us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:56:11.356870 (+ 305us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:56:11.356873 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:56:11.356875 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:56:11.357023 (+ 148us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:56:11.357541 (+ 518us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 02:56:11.357548 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 02:56:11.357813 (+ 265us) client_negotiation.cc:769] Sending connection context
0811 02:56:11.358016 (+ 203us) client_negotiation.cc:240] Negotiation successful
0811 02:56:11.358446 (+ 430us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":267,"thread_start_us":123,"threads_started":1}
I20260811 02:56:11.360579 3138 heartbeater.cc:347] Connected to a master server at 127.12.138.254:35843
I20260811 02:56:11.360864 3138 heartbeater.cc:464] Registering TS with master...
I20260811 02:56:11.361529 3138 heartbeater.cc:511] Master 127.12.138.254:35843 requested a full tablet report, sending...
I20260811 02:56:11.362701 2780 ts_manager.cc:194] Registered new tserver with Master: e2d485f769814847ba1861dcfb5c4318 (127.12.138.194:43359)
I20260811 02:56:11.363353 2780 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.12.138.194@KRBTEST.COM'} at 127.12.138.194:43489
WARNING: no policy specified for kudu/127.12.138.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.138.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.138.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786416707705494-12843-0/krb5kdc/kudu_127.12.138.195.keytab.
Entry for principal kudu/127.12.138.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786416707705494-12843-0/krb5kdc/kudu_127.12.138.195.keytab.
I20260811 02:56:11.384523 12843 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDawEwS/build/debug/bin/kudu
/tmp/dist-test-taskDawEwS/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786416707705494-12843-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786416707705494-12843-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786416707705494-12843-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786416707705494-12843-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.138.195:0
--local_ip_for_outbound_sockets=127.12.138.195
--webserver_interface=127.12.138.195
--webserver_port=0
--tserver_master_addrs=127.12.138.254:35843
--keytab_file=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786416707705494-12843-0/krb5kdc/kudu_127.12.138.195.keytab
--principal=kudu/127.12.138.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.138.212:34411
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786416707705494-12843-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786416707705494-12843-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786416707705494-12843-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 02:56:11.500113 3143 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 02:56:11.500380 3143 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 02:56:11.500422 3143 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 02:56:11.504490 3143 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 02:56:11.504554 3143 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 02:56:11.504645 3143 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.138.195
I20260811 02:56:11.510169 3143 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.138.212:34411
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786416707705494-12843-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786416707705494-12843-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786416707705494-12843-0/krb5kdc/kudu_127.12.138.195.keytab
--principal=kudu/127.12.138.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.138.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786416707705494-12843-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.12.138.195
--webserver_port=0
--tserver_master_addrs=127.12.138.254:35843
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3143
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.138.195
--log_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786416707705494-12843-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 02:09:49 UTC on ccycloud.quasar-fsnqqc.root.comops.site
build id 4579570
I20260811 02:56:11.511430 3143 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 02:56:11.512720 3143 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260811 02:56:11.523036 3143 server_base.cc:1034] running on GCE node
W20260811 02:56:11.523008 3152 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:56:11.522987 3149 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:56:11.523016 3148 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:56:11.523849 3143 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 02:56:11.524509 3143 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 02:56:11.525691 3143 hybrid_clock.cc:648] HybridClock initialized: now 1786416971525646 us; error 58 us; skew 500 ppm
Aug 11 02:56:11 dist-test-slave-9gcw krb5kdc[2730](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416971, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.138.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 02:56:11.528528 3143 init.cc:377] Logged in from keytab as kudu/127.12.138.195@KRBTEST.COM (short username kudu)
I20260811 02:56:11.529474 3143 webserver.cc:460] Webserver started at http://127.12.138.195:38689/ using document root <none> and password file <none>
I20260811 02:56:11.530040 3143 fs_manager.cc:359] Metadata directory not provided
I20260811 02:56:11.530103 3143 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 02:56:11.530339 3143 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 02:56:11.532406 3143 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786416707705494-12843-0/minicluster-data/ts-2/data/instance:
uuid: "54ecd351144b4e969a4d723c7ff023d6"
format_stamp: "Formatted at 2026-08-11 02:56:11 on dist-test-slave-9gcw"
server_key: "e049c2a11d95a86b425d15cd85c39268"
server_key_iv: "8c971660e8516d8d1078a9899eb833cf"
server_key_version: "encryptionkey@0"
I20260811 02:56:11.532982 3143 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786416707705494-12843-0/minicluster-data/ts-2/wal/instance:
uuid: "54ecd351144b4e969a4d723c7ff023d6"
format_stamp: "Formatted at 2026-08-11 02:56:11 on dist-test-slave-9gcw"
server_key: "e049c2a11d95a86b425d15cd85c39268"
server_key_iv: "8c971660e8516d8d1078a9899eb833cf"
server_key_version: "encryptionkey@0"
I20260811 02:56:11.536787 3143 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.001s
I20260811 02:56:11.539361 3160 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 02:56:11.540578 3143 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260811 02:56:11.540702 3143 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786416707705494-12843-0/minicluster-data/ts-2/data,/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786416707705494-12843-0/minicluster-data/ts-2/wal
uuid: "54ecd351144b4e969a4d723c7ff023d6"
format_stamp: "Formatted at 2026-08-11 02:56:11 on dist-test-slave-9gcw"
server_key: "e049c2a11d95a86b425d15cd85c39268"
server_key_iv: "8c971660e8516d8d1078a9899eb833cf"
server_key_version: "encryptionkey@0"
I20260811 02:56:11.540808 3143 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786416707705494-12843-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786416707705494-12843-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786416707705494-12843-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 02:56:11.562755 3143 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 02:56:11.565909 3143 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 02:56:11.566134 3143 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 02:56:11.566797 3143 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 02:56:11.567696 3143 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 02:56:11.567746 3143 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 02:56:11.567816 3143 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 02:56:11.567849 3143 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 02:56:11.577240 3143 rpc_server.cc:307] RPC server started. Bound to: 127.12.138.195:32989
I20260811 02:56:11.577303 3294 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.138.195:32989 every 8 connection(s)
I20260811 02:56:11.578224 3143 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786416707705494-12843-0/minicluster-data/ts-2/data/info.pb
I20260811 02:56:11.582458 12843 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDawEwS/build/debug/bin/kudu as pid 3143
I20260811 02:56:11.582579 12843 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786416707705494-12843-0/minicluster-data/ts-2/wal/instance
I20260811 02:56:11.582954 12843 external_mini_cluster.cc:1428] Setting key ca63e88b37bf824168773fe7afe9b842
Aug 11 02:56:11 dist-test-slave-9gcw krb5kdc[2730](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416971, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.138.195@KRBTEST.COM for kudu/127.12.138.254@KRBTEST.COM
I20260811 02:56:11.594054 2822 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:56:11.580097 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.138.195:55073 (local address 127.12.138.254:35843)
0811 02:56:11.580264 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0811 02:56:11.580270 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 02:56:11.581233 (+ 963us) server_negotiation.cc:407] Connection header received
0811 02:56:11.582866 (+ 1633us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:56:11.582869 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:56:11.582935 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:56:11.583054 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:56:11.584667 (+ 1613us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:56:11.585472 (+ 805us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:56:11.586925 (+ 1453us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:56:11.587176 (+ 251us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:56:11.587301 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:56:11.590118 (+ 2817us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:56:11.590121 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:56:11.590123 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:56:11.590150 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:56:11.591749 (+ 1599us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:56:11.592322 (+ 573us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:56:11.592324 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:56:11.592326 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:56:11.592407 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:56:11.592865 (+ 458us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:56:11.592868 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:56:11.592869 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:56:11.593057 (+ 188us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:56:11.593135 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0811 02:56:11.593799 (+ 664us) server_negotiation.cc:299] Negotiation successful
0811 02:56:11.593916 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260811 02:56:11.595168 3298 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:56:11.580510 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.138.254:35843 (local address 127.12.138.195:55073)
0811 02:56:11.581031 (+ 521us) negotiation.cc:107] Waiting for socket to connect
0811 02:56:11.581078 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0811 02:56:11.582591 (+ 1513us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:56:11.583259 (+ 668us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:56:11.583297 (+ 38us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:56:11.583773 (+ 476us) client_negotiation.cc:189] Negotiated authn=SASL
0811 02:56:11.584496 (+ 723us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:56:11.584515 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:56:11.585616 (+ 1101us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:56:11.585620 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:56:11.586763 (+ 1143us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:56:11.586780 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:56:11.587341 (+ 561us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:56:11.587346 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:56:11.587541 (+ 195us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:56:11.588137 (+ 596us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 02:56:11.588156 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0811 02:56:11.589921 (+ 1765us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 02:56:11.591880 (+ 1959us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:56:11.591887 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:56:11.591901 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:56:11.592219 (+ 318us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:56:11.592535 (+ 316us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:56:11.592538 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:56:11.592539 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:56:11.592754 (+ 215us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:56:11.593228 (+ 474us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 02:56:11.593235 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 02:56:11.593527 (+ 292us) client_negotiation.cc:769] Sending connection context
0811 02:56:11.593724 (+ 197us) client_negotiation.cc:240] Negotiation successful
0811 02:56:11.594038 (+ 314us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":294,"thread_start_us":112,"threads_started":1}
I20260811 02:56:11.596473 3295 heartbeater.cc:347] Connected to a master server at 127.12.138.254:35843
I20260811 02:56:11.596766 3295 heartbeater.cc:464] Registering TS with master...
I20260811 02:56:11.597492 3295 heartbeater.cc:511] Master 127.12.138.254:35843 requested a full tablet report, sending...
I20260811 02:56:11.598539 2779 ts_manager.cc:194] Registered new tserver with Master: 54ecd351144b4e969a4d723c7ff023d6 (127.12.138.195:32989)
I20260811 02:56:11.599210 2779 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.12.138.195@KRBTEST.COM'} at 127.12.138.195:55073
I20260811 02:56:11.609401 12843 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 02:56:11.619632 2822 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:56:11.611463 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45110 (local address 127.12.138.254:35843)
0811 02:56:11.611621 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0811 02:56:11.611626 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 02:56:11.611817 (+ 191us) server_negotiation.cc:407] Connection header received
0811 02:56:11.611985 (+ 168us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:56:11.611988 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:56:11.612039 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:56:11.612120 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:56:11.612894 (+ 774us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:56:11.613506 (+ 612us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:56:11.614384 (+ 878us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:56:11.614606 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:56:11.614712 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:56:11.616354 (+ 1642us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:56:11.616364 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:56:11.616366 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:56:11.616390 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:56:11.617873 (+ 1483us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:56:11.618330 (+ 457us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:56:11.618335 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:56:11.618336 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:56:11.618421 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:56:11.618701 (+ 280us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:56:11.618704 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:56:11.618706 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:56:11.618943 (+ 237us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:56:11.619087 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0811 02:56:11.619338 (+ 251us) server_negotiation.cc:299] Negotiation successful
0811 02:56:11.619458 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260811 02:56:11.622807 2779 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45110:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 02:56:11.626098 2779 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 02:56:11.642283 3312 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:56:11.635399 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60570 (local address 127.12.138.195:32989)
0811 02:56:11.637414 (+ 2015us) server_negotiation.cc:206] Beginning negotiation
0811 02:56:11.637443 (+ 29us) server_negotiation.cc:399] Waiting for connection header
0811 02:56:11.637468 (+ 25us) server_negotiation.cc:407] Connection header received
0811 02:56:11.637540 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:56:11.637545 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:56:11.637723 (+ 178us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:56:11.637877 (+ 154us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 02:56:11.638102 (+ 225us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:56:11.638919 (+ 817us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:56:11.640652 (+ 1733us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:56:11.641527 (+ 875us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:56:11.641702 (+ 175us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:56:11.641808 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0811 02:56:11.641994 (+ 186us) server_negotiation.cc:299] Negotiation successful
0811 02:56:11.642104 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1929,"thread_start_us":189,"threads_started":1}
I20260811 02:56:11.642287 3309 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:56:11.635039 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.138.195:32989 (local address 127.0.0.1:60570)
0811 02:56:11.635495 (+ 456us) negotiation.cc:107] Waiting for socket to connect
0811 02:56:11.635556 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0811 02:56:11.635699 (+ 143us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:56:11.637862 (+ 2163us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:56:11.637865 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:56:11.637881 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 02:56:11.637969 (+ 88us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:56:11.637973 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:56:11.639070 (+ 1097us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:56:11.639073 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:56:11.640488 (+ 1415us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:56:11.640499 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:56:11.641656 (+ 1157us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:56:11.641660 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:56:11.641782 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:56:11.641828 (+ 46us) client_negotiation.cc:769] Sending connection context
0811 02:56:11.641957 (+ 129us) client_negotiation.cc:240] Negotiation successful
0811 02:56:11.642091 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":253,"mutex_wait_us":156,"thread_start_us":109,"threads_started":1}
I20260811 02:56:11.642863 3315 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:56:11.636136 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.138.193:34809 (local address 127.0.0.1:36636)
0811 02:56:11.636381 (+ 245us) negotiation.cc:107] Waiting for socket to connect
0811 02:56:11.636394 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 02:56:11.636476 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:56:11.637096 (+ 620us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:56:11.637098 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:56:11.637134 (+ 36us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 02:56:11.637357 (+ 223us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:56:11.637365 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:56:11.639297 (+ 1932us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:56:11.639300 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:56:11.641095 (+ 1795us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:56:11.641104 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:56:11.642416 (+ 1312us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:56:11.642421 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:56:11.642517 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:56:11.642540 (+ 23us) client_negotiation.cc:769] Sending connection context
0811 02:56:11.642614 (+ 74us) client_negotiation.cc:240] Negotiation successful
0811 02:56:11.642691 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":189,"mutex_wait_us":83,"spinlock_wait_cycles":16768,"thread_start_us":116,"threads_started":1}
I20260811 02:56:11.643079 3314 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:56:11.635039 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.138.194:43359 (local address 127.0.0.1:55592)
0811 02:56:11.636016 (+ 977us) negotiation.cc:107] Waiting for socket to connect
0811 02:56:11.636028 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 02:56:11.636131 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:56:11.637089 (+ 958us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:56:11.637096 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:56:11.637133 (+ 37us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 02:56:11.637371 (+ 238us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:56:11.637377 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:56:11.638851 (+ 1474us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:56:11.638855 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:56:11.640463 (+ 1608us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:56:11.640475 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:56:11.642674 (+ 2199us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:56:11.642678 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:56:11.642770 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:56:11.642791 (+ 21us) client_negotiation.cc:769] Sending connection context
0811 02:56:11.642862 (+ 71us) client_negotiation.cc:240] Negotiation successful
0811 02:56:11.642935 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":907,"mutex_wait_us":102,"spinlock_wait_cycles":4352,"thread_start_us":786,"threads_started":1}
I20260811 02:56:11.643384 3316 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:56:11.636337 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36636 (local address 127.12.138.193:34809)
0811 02:56:11.636731 (+ 394us) server_negotiation.cc:206] Beginning negotiation
0811 02:56:11.636737 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 02:56:11.636757 (+ 20us) server_negotiation.cc:407] Connection header received
0811 02:56:11.636823 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:56:11.636828 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:56:11.636982 (+ 154us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:56:11.637112 (+ 130us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 02:56:11.638045 (+ 933us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:56:11.639147 (+ 1102us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:56:11.641319 (+ 2172us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:56:11.642107 (+ 788us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:56:11.642226 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:56:11.642348 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0811 02:56:11.643094 (+ 746us) server_negotiation.cc:299] Negotiation successful
0811 02:56:11.643226 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":296,"thread_start_us":102,"threads_started":1}
I20260811 02:56:11.643603 3310 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:56:11.635366 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55592 (local address 127.12.138.194:43359)
0811 02:56:11.636540 (+ 1174us) server_negotiation.cc:206] Beginning negotiation
0811 02:56:11.636553 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0811 02:56:11.636579 (+ 26us) server_negotiation.cc:407] Connection header received
0811 02:56:11.636666 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:56:11.636671 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:56:11.636891 (+ 220us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:56:11.637062 (+ 171us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 02:56:11.637488 (+ 426us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:56:11.638677 (+ 1189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:56:11.640599 (+ 1922us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:56:11.641527 (+ 928us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:56:11.641661 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:56:11.641839 (+ 178us) server_negotiation.cc:1035] Waiting for connection context
0811 02:56:11.643352 (+ 1513us) server_negotiation.cc:299] Negotiation successful
0811 02:56:11.643461 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1065,"thread_start_us":149,"threads_started":1}
I20260811 02:56:11.645725 3215 tablet_service.cc:1467] Processing CreateTablet for tablet 2dce9862b85b401796bf57840306ccac (DEFAULT_TABLE table=test-table [id=f98b5aedc3224e7383888385628b48a8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 02:56:11.645985 2918 tablet_service.cc:1467] Processing CreateTablet for tablet 2dce9862b85b401796bf57840306ccac (DEFAULT_TABLE table=test-table [id=f98b5aedc3224e7383888385628b48a8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 02:56:11.646992 3215 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2dce9862b85b401796bf57840306ccac. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 02:56:11.647032 2918 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2dce9862b85b401796bf57840306ccac. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 02:56:11.647837 3061 tablet_service.cc:1467] Processing CreateTablet for tablet 2dce9862b85b401796bf57840306ccac (DEFAULT_TABLE table=test-table [id=f98b5aedc3224e7383888385628b48a8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 02:56:11.648829 3061 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2dce9862b85b401796bf57840306ccac. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 02:56:11.654101 3317 tablet_bootstrap.cc:492] T 2dce9862b85b401796bf57840306ccac P 54ecd351144b4e969a4d723c7ff023d6: Bootstrap starting.
I20260811 02:56:11.655867 3320 tablet_bootstrap.cc:492] T 2dce9862b85b401796bf57840306ccac P e2d485f769814847ba1861dcfb5c4318: Bootstrap starting.
I20260811 02:56:11.656147 3317 tablet_bootstrap.cc:654] T 2dce9862b85b401796bf57840306ccac P 54ecd351144b4e969a4d723c7ff023d6: Neither blocks nor log segments found. Creating new log.
I20260811 02:56:11.656440 3318 tablet_bootstrap.cc:492] T 2dce9862b85b401796bf57840306ccac P 566fc2a50bfa404ebfa81c06ec9a85c9: Bootstrap starting.
I20260811 02:56:11.657655 3317 log.cc:824] T 2dce9862b85b401796bf57840306ccac P 54ecd351144b4e969a4d723c7ff023d6: Log is configured to *not* fsync() on all Append() calls
I20260811 02:56:11.658180 3320 tablet_bootstrap.cc:654] T 2dce9862b85b401796bf57840306ccac P e2d485f769814847ba1861dcfb5c4318: Neither blocks nor log segments found. Creating new log.
I20260811 02:56:11.658954 3318 tablet_bootstrap.cc:654] T 2dce9862b85b401796bf57840306ccac P 566fc2a50bfa404ebfa81c06ec9a85c9: Neither blocks nor log segments found. Creating new log.
I20260811 02:56:11.659431 3320 log.cc:824] T 2dce9862b85b401796bf57840306ccac P e2d485f769814847ba1861dcfb5c4318: Log is configured to *not* fsync() on all Append() calls
I20260811 02:56:11.659735 3318 log.cc:824] T 2dce9862b85b401796bf57840306ccac P 566fc2a50bfa404ebfa81c06ec9a85c9: Log is configured to *not* fsync() on all Append() calls
I20260811 02:56:11.660104 3317 tablet_bootstrap.cc:492] T 2dce9862b85b401796bf57840306ccac P 54ecd351144b4e969a4d723c7ff023d6: No bootstrap required, opened a new log
I20260811 02:56:11.660301 3317 ts_tablet_manager.cc:1397] T 2dce9862b85b401796bf57840306ccac P 54ecd351144b4e969a4d723c7ff023d6: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260811 02:56:11.661501 3320 tablet_bootstrap.cc:492] T 2dce9862b85b401796bf57840306ccac P e2d485f769814847ba1861dcfb5c4318: No bootstrap required, opened a new log
I20260811 02:56:11.661689 3320 ts_tablet_manager.cc:1397] T 2dce9862b85b401796bf57840306ccac P e2d485f769814847ba1861dcfb5c4318: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260811 02:56:11.662863 3318 tablet_bootstrap.cc:492] T 2dce9862b85b401796bf57840306ccac P 566fc2a50bfa404ebfa81c06ec9a85c9: No bootstrap required, opened a new log
I20260811 02:56:11.663043 3318 ts_tablet_manager.cc:1397] T 2dce9862b85b401796bf57840306ccac P 566fc2a50bfa404ebfa81c06ec9a85c9: Time spent bootstrapping tablet: real 0.007s user 0.004s sys 0.000s
I20260811 02:56:11.664007 3317 raft_consensus.cc:348] T 2dce9862b85b401796bf57840306ccac P 54ecd351144b4e969a4d723c7ff023d6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2d485f769814847ba1861dcfb5c4318" member_type: VOTER last_known_addr { host: "127.12.138.194" port: 43359 } } peers { permanent_uuid: "54ecd351144b4e969a4d723c7ff023d6" member_type: VOTER last_known_addr { host: "127.12.138.195" port: 32989 } } peers { permanent_uuid: "566fc2a50bfa404ebfa81c06ec9a85c9" member_type: VOTER last_known_addr { host: "127.12.138.193" port: 34809 } }
I20260811 02:56:11.664310 3317 raft_consensus.cc:374] T 2dce9862b85b401796bf57840306ccac P 54ecd351144b4e969a4d723c7ff023d6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 02:56:11.664362 3317 raft_consensus.cc:729] T 2dce9862b85b401796bf57840306ccac P 54ecd351144b4e969a4d723c7ff023d6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 54ecd351144b4e969a4d723c7ff023d6, State: Initialized, Role: FOLLOWER
I20260811 02:56:11.664848 3317 consensus_queue.cc:260] T 2dce9862b85b401796bf57840306ccac P 54ecd351144b4e969a4d723c7ff023d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2d485f769814847ba1861dcfb5c4318" member_type: VOTER last_known_addr { host: "127.12.138.194" port: 43359 } } peers { permanent_uuid: "54ecd351144b4e969a4d723c7ff023d6" member_type: VOTER last_known_addr { host: "127.12.138.195" port: 32989 } } peers { permanent_uuid: "566fc2a50bfa404ebfa81c06ec9a85c9" member_type: VOTER last_known_addr { host: "127.12.138.193" port: 34809 } }
I20260811 02:56:11.665144 3320 raft_consensus.cc:348] T 2dce9862b85b401796bf57840306ccac P e2d485f769814847ba1861dcfb5c4318 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2d485f769814847ba1861dcfb5c4318" member_type: VOTER last_known_addr { host: "127.12.138.194" port: 43359 } } peers { permanent_uuid: "54ecd351144b4e969a4d723c7ff023d6" member_type: VOTER last_known_addr { host: "127.12.138.195" port: 32989 } } peers { permanent_uuid: "566fc2a50bfa404ebfa81c06ec9a85c9" member_type: VOTER last_known_addr { host: "127.12.138.193" port: 34809 } }
I20260811 02:56:11.665460 3320 raft_consensus.cc:374] T 2dce9862b85b401796bf57840306ccac P e2d485f769814847ba1861dcfb5c4318 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 02:56:11.665535 3320 raft_consensus.cc:729] T 2dce9862b85b401796bf57840306ccac P e2d485f769814847ba1861dcfb5c4318 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e2d485f769814847ba1861dcfb5c4318, State: Initialized, Role: FOLLOWER
I20260811 02:56:11.666023 3317 ts_tablet_manager.cc:1428] T 2dce9862b85b401796bf57840306ccac P 54ecd351144b4e969a4d723c7ff023d6: Time spent starting tablet: real 0.006s user 0.003s sys 0.004s
I20260811 02:56:11.666047 3295 heartbeater.cc:503] Master 127.12.138.254:35843 was elected leader, sending a full tablet report...
I20260811 02:56:11.665994 3320 consensus_queue.cc:260] T 2dce9862b85b401796bf57840306ccac P e2d485f769814847ba1861dcfb5c4318 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2d485f769814847ba1861dcfb5c4318" member_type: VOTER last_known_addr { host: "127.12.138.194" port: 43359 } } peers { permanent_uuid: "54ecd351144b4e969a4d723c7ff023d6" member_type: VOTER last_known_addr { host: "127.12.138.195" port: 32989 } } peers { permanent_uuid: "566fc2a50bfa404ebfa81c06ec9a85c9" member_type: VOTER last_known_addr { host: "127.12.138.193" port: 34809 } }
I20260811 02:56:11.666724 3138 heartbeater.cc:503] Master 127.12.138.254:35843 was elected leader, sending a full tablet report...
I20260811 02:56:11.666915 3320 ts_tablet_manager.cc:1428] T 2dce9862b85b401796bf57840306ccac P e2d485f769814847ba1861dcfb5c4318: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260811 02:56:11.668624 3318 raft_consensus.cc:348] T 2dce9862b85b401796bf57840306ccac P 566fc2a50bfa404ebfa81c06ec9a85c9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2d485f769814847ba1861dcfb5c4318" member_type: VOTER last_known_addr { host: "127.12.138.194" port: 43359 } } peers { permanent_uuid: "54ecd351144b4e969a4d723c7ff023d6" member_type: VOTER last_known_addr { host: "127.12.138.195" port: 32989 } } peers { permanent_uuid: "566fc2a50bfa404ebfa81c06ec9a85c9" member_type: VOTER last_known_addr { host: "127.12.138.193" port: 34809 } }
I20260811 02:56:11.668942 3318 raft_consensus.cc:374] T 2dce9862b85b401796bf57840306ccac P 566fc2a50bfa404ebfa81c06ec9a85c9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 02:56:11.669023 3318 raft_consensus.cc:729] T 2dce9862b85b401796bf57840306ccac P 566fc2a50bfa404ebfa81c06ec9a85c9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 566fc2a50bfa404ebfa81c06ec9a85c9, State: Initialized, Role: FOLLOWER
I20260811 02:56:11.669512 3318 consensus_queue.cc:260] T 2dce9862b85b401796bf57840306ccac P 566fc2a50bfa404ebfa81c06ec9a85c9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2d485f769814847ba1861dcfb5c4318" member_type: VOTER last_known_addr { host: "127.12.138.194" port: 43359 } } peers { permanent_uuid: "54ecd351144b4e969a4d723c7ff023d6" member_type: VOTER last_known_addr { host: "127.12.138.195" port: 32989 } } peers { permanent_uuid: "566fc2a50bfa404ebfa81c06ec9a85c9" member_type: VOTER last_known_addr { host: "127.12.138.193" port: 34809 } }
I20260811 02:56:11.670157 2989 heartbeater.cc:503] Master 127.12.138.254:35843 was elected leader, sending a full tablet report...
I20260811 02:56:11.670516 3318 ts_tablet_manager.cc:1428] T 2dce9862b85b401796bf57840306ccac P 566fc2a50bfa404ebfa81c06ec9a85c9: Time spent starting tablet: real 0.007s user 0.003s sys 0.004s
I20260811 02:56:11.680505 3324 raft_consensus.cc:482] T 2dce9862b85b401796bf57840306ccac P 54ecd351144b4e969a4d723c7ff023d6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 02:56:11.680742 3324 raft_consensus.cc:504] T 2dce9862b85b401796bf57840306ccac P 54ecd351144b4e969a4d723c7ff023d6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2d485f769814847ba1861dcfb5c4318" member_type: VOTER last_known_addr { host: "127.12.138.194" port: 43359 } } peers { permanent_uuid: "54ecd351144b4e969a4d723c7ff023d6" member_type: VOTER last_known_addr { host: "127.12.138.195" port: 32989 } } peers { permanent_uuid: "566fc2a50bfa404ebfa81c06ec9a85c9" member_type: VOTER last_known_addr { host: "127.12.138.193" port: 34809 } }
I20260811 02:56:11.682004 3324 leader_election.cc:288] T 2dce9862b85b401796bf57840306ccac P 54ecd351144b4e969a4d723c7ff023d6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e2d485f769814847ba1861dcfb5c4318 (127.12.138.194:43359), 566fc2a50bfa404ebfa81c06ec9a85c9 (127.12.138.193:34809)
I20260811 02:56:11.686268 3298 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:56:11.682235 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.138.194:43359 (local address 127.12.138.195:48529)
0811 02:56:11.682404 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0811 02:56:11.682422 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 02:56:11.682548 (+ 126us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:56:11.682946 (+ 398us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:56:11.682952 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:56:11.682979 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 02:56:11.683128 (+ 149us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:56:11.683134 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:56:11.684151 (+ 1017us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:56:11.684175 (+ 24us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:56:11.685126 (+ 951us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:56:11.685136 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:56:11.685823 (+ 687us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:56:11.685826 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:56:11.685931 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:56:11.685953 (+ 22us) client_negotiation.cc:769] Sending connection context
0811 02:56:11.686046 (+ 93us) client_negotiation.cc:240] Negotiation successful
0811 02:56:11.686130 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":59,"mutex_wait_us":22,"spinlock_wait_cycles":9600}
I20260811 02:56:11.686268 3310 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:56:11.682306 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.138.195:48529 (local address 127.12.138.194:43359)
0811 02:56:11.682435 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0811 02:56:11.682442 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 02:56:11.682487 (+ 45us) server_negotiation.cc:407] Connection header received
0811 02:56:11.682683 (+ 196us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:56:11.682689 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:56:11.682762 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:56:11.682848 (+ 86us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 02:56:11.683258 (+ 410us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:56:11.684038 (+ 780us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:56:11.685306 (+ 1268us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:56:11.685703 (+ 397us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:56:11.685835 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:56:11.685892 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0811 02:56:11.686063 (+ 171us) server_negotiation.cc:299] Negotiation successful
0811 02:56:11.686134 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260811 02:56:11.686434 3316 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:56:11.682691 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.138.195:37071 (local address 127.12.138.193:34809)
0811 02:56:11.682870 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0811 02:56:11.682874 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 02:56:11.682998 (+ 124us) server_negotiation.cc:407] Connection header received
0811 02:56:11.683127 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:56:11.683132 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:56:11.683188 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:56:11.683584 (+ 396us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 02:56:11.683788 (+ 204us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:56:11.684289 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:56:11.685450 (+ 1161us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:56:11.685872 (+ 422us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:56:11.686002 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:56:11.686050 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0811 02:56:11.686252 (+ 202us) server_negotiation.cc:299] Negotiation successful
0811 02:56:11.686329 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":100}
I20260811 02:56:11.686671 3327 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:56:11.682541 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.138.193:34809 (local address 127.12.138.195:37071)
0811 02:56:11.682909 (+ 368us) negotiation.cc:107] Waiting for socket to connect
0811 02:56:11.682924 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 02:56:11.683024 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:56:11.683282 (+ 258us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:56:11.683285 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:56:11.683302 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 02:56:11.683417 (+ 115us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:56:11.683422 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:56:11.684454 (+ 1032us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:56:11.684460 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:56:11.685325 (+ 865us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:56:11.685335 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:56:11.685997 (+ 662us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:56:11.686000 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:56:11.686125 (+ 125us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:56:11.686143 (+ 18us) client_negotiation.cc:769] Sending connection context
0811 02:56:11.686231 (+ 88us) client_negotiation.cc:240] Negotiation successful
0811 02:56:11.686301 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":299,"mutex_wait_us":29,"thread_start_us":151,"threads_started":1}
I20260811 02:56:11.687129 3082 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2dce9862b85b401796bf57840306ccac" candidate_uuid: "54ecd351144b4e969a4d723c7ff023d6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e2d485f769814847ba1861dcfb5c4318" is_pre_election: true
I20260811 02:56:11.687278 2941 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2dce9862b85b401796bf57840306ccac" candidate_uuid: "54ecd351144b4e969a4d723c7ff023d6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "566fc2a50bfa404ebfa81c06ec9a85c9" is_pre_election: true
I20260811 02:56:11.687477 3082 raft_consensus.cc:2454] T 2dce9862b85b401796bf57840306ccac P e2d485f769814847ba1861dcfb5c4318 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 54ecd351144b4e969a4d723c7ff023d6 in term 0.
I20260811 02:56:11.687510 2941 raft_consensus.cc:2454] T 2dce9862b85b401796bf57840306ccac P 566fc2a50bfa404ebfa81c06ec9a85c9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 54ecd351144b4e969a4d723c7ff023d6 in term 0.
I20260811 02:56:11.687965 3164 leader_election.cc:302] T 2dce9862b85b401796bf57840306ccac P 54ecd351144b4e969a4d723c7ff023d6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 54ecd351144b4e969a4d723c7ff023d6, e2d485f769814847ba1861dcfb5c4318; no voters:
I20260811 02:56:11.688202 3324 raft_consensus.cc:2781] T 2dce9862b85b401796bf57840306ccac P 54ecd351144b4e969a4d723c7ff023d6 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 02:56:11.688308 3324 raft_consensus.cc:482] T 2dce9862b85b401796bf57840306ccac P 54ecd351144b4e969a4d723c7ff023d6 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 02:56:11.688371 3324 raft_consensus.cc:3037] T 2dce9862b85b401796bf57840306ccac P 54ecd351144b4e969a4d723c7ff023d6 [term 0 FOLLOWER]: Advancing to term 1
I20260811 02:56:11.689519 3324 raft_consensus.cc:504] T 2dce9862b85b401796bf57840306ccac P 54ecd351144b4e969a4d723c7ff023d6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2d485f769814847ba1861dcfb5c4318" member_type: VOTER last_known_addr { host: "127.12.138.194" port: 43359 } } peers { permanent_uuid: "54ecd351144b4e969a4d723c7ff023d6" member_type: VOTER last_known_addr { host: "127.12.138.195" port: 32989 } } peers { permanent_uuid: "566fc2a50bfa404ebfa81c06ec9a85c9" member_type: VOTER last_known_addr { host: "127.12.138.193" port: 34809 } }
I20260811 02:56:11.689898 3324 leader_election.cc:288] T 2dce9862b85b401796bf57840306ccac P 54ecd351144b4e969a4d723c7ff023d6 [CANDIDATE]: Term 1 election: Requested vote from peers e2d485f769814847ba1861dcfb5c4318 (127.12.138.194:43359), 566fc2a50bfa404ebfa81c06ec9a85c9 (127.12.138.193:34809)
I20260811 02:56:11.690232 3082 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2dce9862b85b401796bf57840306ccac" candidate_uuid: "54ecd351144b4e969a4d723c7ff023d6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e2d485f769814847ba1861dcfb5c4318"
I20260811 02:56:11.690301 2941 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2dce9862b85b401796bf57840306ccac" candidate_uuid: "54ecd351144b4e969a4d723c7ff023d6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "566fc2a50bfa404ebfa81c06ec9a85c9"
I20260811 02:56:11.690408 2941 raft_consensus.cc:3037] T 2dce9862b85b401796bf57840306ccac P 566fc2a50bfa404ebfa81c06ec9a85c9 [term 0 FOLLOWER]: Advancing to term 1
I20260811 02:56:11.690434 3082 raft_consensus.cc:3037] T 2dce9862b85b401796bf57840306ccac P e2d485f769814847ba1861dcfb5c4318 [term 0 FOLLOWER]: Advancing to term 1
I20260811 02:56:11.691761 2941 raft_consensus.cc:2454] T 2dce9862b85b401796bf57840306ccac P 566fc2a50bfa404ebfa81c06ec9a85c9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 54ecd351144b4e969a4d723c7ff023d6 in term 1.
I20260811 02:56:11.691754 3082 raft_consensus.cc:2454] T 2dce9862b85b401796bf57840306ccac P e2d485f769814847ba1861dcfb5c4318 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 54ecd351144b4e969a4d723c7ff023d6 in term 1.
I20260811 02:56:11.692046 3164 leader_election.cc:302] T 2dce9862b85b401796bf57840306ccac P 54ecd351144b4e969a4d723c7ff023d6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 54ecd351144b4e969a4d723c7ff023d6, 566fc2a50bfa404ebfa81c06ec9a85c9; no voters:
I20260811 02:56:11.692196 3324 raft_consensus.cc:2781] T 2dce9862b85b401796bf57840306ccac P 54ecd351144b4e969a4d723c7ff023d6 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 02:56:11.692440 3324 raft_consensus.cc:686] T 2dce9862b85b401796bf57840306ccac P 54ecd351144b4e969a4d723c7ff023d6 [term 1 LEADER]: Becoming Leader. State: Replica: 54ecd351144b4e969a4d723c7ff023d6, State: Running, Role: LEADER
I20260811 02:56:11.692858 3324 consensus_queue.cc:237] T 2dce9862b85b401796bf57840306ccac P 54ecd351144b4e969a4d723c7ff023d6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2d485f769814847ba1861dcfb5c4318" member_type: VOTER last_known_addr { host: "127.12.138.194" port: 43359 } } peers { permanent_uuid: "54ecd351144b4e969a4d723c7ff023d6" member_type: VOTER last_known_addr { host: "127.12.138.195" port: 32989 } } peers { permanent_uuid: "566fc2a50bfa404ebfa81c06ec9a85c9" member_type: VOTER last_known_addr { host: "127.12.138.193" port: 34809 } }
I20260811 02:56:11.696206 2779 catalog_manager.cc:5495] T 2dce9862b85b401796bf57840306ccac P 54ecd351144b4e969a4d723c7ff023d6 reported cstate change: term changed from 0 to 1, leader changed from <none> to 54ecd351144b4e969a4d723c7ff023d6 (127.12.138.195). New cstate: current_term: 1 leader_uuid: "54ecd351144b4e969a4d723c7ff023d6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2d485f769814847ba1861dcfb5c4318" member_type: VOTER last_known_addr { host: "127.12.138.194" port: 43359 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "54ecd351144b4e969a4d723c7ff023d6" member_type: VOTER last_known_addr { host: "127.12.138.195" port: 32989 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "566fc2a50bfa404ebfa81c06ec9a85c9" member_type: VOTER last_known_addr { host: "127.12.138.193" port: 34809 } health_report { overall_health: UNKNOWN } } }
I20260811 02:56:11.720124 3312 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:56:11.716655 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60580 (local address 127.12.138.195:32989)
0811 02:56:11.716787 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0811 02:56:11.716790 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 02:56:11.716803 (+ 13us) server_negotiation.cc:407] Connection header received
0811 02:56:11.716922 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:56:11.716924 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:56:11.716981 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:56:11.717083 (+ 102us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 02:56:11.717353 (+ 270us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:56:11.717906 (+ 553us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:56:11.718759 (+ 853us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:56:11.718973 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:56:11.719052 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:56:11.719363 (+ 311us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 02:56:11.719812 (+ 449us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 02:56:11.719906 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0811 02:56:11.719969 (+ 63us) server_negotiation.cc:299] Negotiation successful
0811 02:56:11.720018 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260811 02:56:11.726236 2941 raft_consensus.cc:1261] T 2dce9862b85b401796bf57840306ccac P 566fc2a50bfa404ebfa81c06ec9a85c9 [term 1 FOLLOWER]: Refusing update from remote peer 54ecd351144b4e969a4d723c7ff023d6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 02:56:11.726254 3082 raft_consensus.cc:1261] T 2dce9862b85b401796bf57840306ccac P e2d485f769814847ba1861dcfb5c4318 [term 1 FOLLOWER]: Refusing update from remote peer 54ecd351144b4e969a4d723c7ff023d6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 02:56:11.726953 3329 consensus_queue.cc:1035] T 2dce9862b85b401796bf57840306ccac P 54ecd351144b4e969a4d723c7ff023d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "566fc2a50bfa404ebfa81c06ec9a85c9" member_type: VOTER last_known_addr { host: "127.12.138.193" port: 34809 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 02:56:11.727149 3324 consensus_queue.cc:1035] T 2dce9862b85b401796bf57840306ccac P 54ecd351144b4e969a4d723c7ff023d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e2d485f769814847ba1861dcfb5c4318" member_type: VOTER last_known_addr { host: "127.12.138.194" port: 43359 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 02:56:11.736317 3334 mvcc.cc:205] Tried to move back new op lower bound from 7317163916182650880 to 7317163916056813568. Current Snapshot: MvccSnapshot[applied={T|T < 7317163916182650880}]
I20260811 02:56:11.743642 2780 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45110:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 02:56:11.743872 2780 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45110:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 02:56:11.746414 2780 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 528aa09ab93a460dbdf08dccfd86354f: Sending DeleteTablet for 3 replicas of tablet 2dce9862b85b401796bf57840306ccac
I20260811 02:56:11.747242 3215 tablet_service.cc:1514] Processing DeleteTablet for tablet 2dce9862b85b401796bf57840306ccac with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 02:56:11 UTC) from {username='kudu', principal='kudu/127.12.138.254@KRBTEST.COM'} at 127.0.0.1:60570
I20260811 02:56:11.747306 3061 tablet_service.cc:1514] Processing DeleteTablet for tablet 2dce9862b85b401796bf57840306ccac with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 02:56:11 UTC) from {username='kudu', principal='kudu/127.12.138.254@KRBTEST.COM'} at 127.0.0.1:55592
I20260811 02:56:11.747480 2918 tablet_service.cc:1514] Processing DeleteTablet for tablet 2dce9862b85b401796bf57840306ccac with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 02:56:11 UTC) from {username='kudu', principal='kudu/127.12.138.254@KRBTEST.COM'} at 127.0.0.1:36636
I20260811 02:56:11.747929 12843 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDawEwS/build/debug/bin/kudu with pid 2849
I20260811 02:56:11.747947 3341 tablet_replica.cc:330] T 2dce9862b85b401796bf57840306ccac P 54ecd351144b4e969a4d723c7ff023d6: stopping tablet replica
I20260811 02:56:11.753742 3341 raft_consensus.cc:2229] T 2dce9862b85b401796bf57840306ccac P 54ecd351144b4e969a4d723c7ff023d6 [term 1 LEADER]: Raft consensus shutting down.
I20260811 02:56:11.754129 3341 raft_consensus.cc:2258] T 2dce9862b85b401796bf57840306ccac P 54ecd351144b4e969a4d723c7ff023d6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 02:56:11.754626 3342 tablet_replica.cc:330] T 2dce9862b85b401796bf57840306ccac P e2d485f769814847ba1861dcfb5c4318: stopping tablet replica
I20260811 02:56:11.754987 3342 raft_consensus.cc:2229] T 2dce9862b85b401796bf57840306ccac P e2d485f769814847ba1861dcfb5c4318 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 02:56:11.755410 3342 raft_consensus.cc:2258] T 2dce9862b85b401796bf57840306ccac P e2d485f769814847ba1861dcfb5c4318 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 02:56:11.757563 3341 ts_tablet_manager.cc:1905] T 2dce9862b85b401796bf57840306ccac P 54ecd351144b4e969a4d723c7ff023d6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 02:56:11.758353 3342 ts_tablet_manager.cc:1905] T 2dce9862b85b401796bf57840306ccac P e2d485f769814847ba1861dcfb5c4318: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 02:56:11.760434 3342 ts_tablet_manager.cc:1918] T 2dce9862b85b401796bf57840306ccac P e2d485f769814847ba1861dcfb5c4318: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 02:56:11.760556 3342 log.cc:1196] T 2dce9862b85b401796bf57840306ccac P e2d485f769814847ba1861dcfb5c4318: Deleting WAL directory at /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786416707705494-12843-0/minicluster-data/ts-1/wal/wals/2dce9862b85b401796bf57840306ccac
I20260811 02:56:11.760550 3341 ts_tablet_manager.cc:1918] T 2dce9862b85b401796bf57840306ccac P 54ecd351144b4e969a4d723c7ff023d6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 02:56:11.760643 3341 log.cc:1196] T 2dce9862b85b401796bf57840306ccac P 54ecd351144b4e969a4d723c7ff023d6: Deleting WAL directory at /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786416707705494-12843-0/minicluster-data/ts-2/wal/wals/2dce9862b85b401796bf57840306ccac
I20260811 02:56:11.760922 3342 ts_tablet_manager.cc:1939] T 2dce9862b85b401796bf57840306ccac P e2d485f769814847ba1861dcfb5c4318: Deleting consensus metadata
I20260811 02:56:11.761030 3341 ts_tablet_manager.cc:1939] T 2dce9862b85b401796bf57840306ccac P 54ecd351144b4e969a4d723c7ff023d6: Deleting consensus metadata
I20260811 02:56:11.762344 2765 catalog_manager.cc:4841] TS e2d485f769814847ba1861dcfb5c4318 (127.12.138.194:43359): tablet 2dce9862b85b401796bf57840306ccac (table test-table [id=f98b5aedc3224e7383888385628b48a8]) successfully deleted
I20260811 02:56:11.762626 2763 catalog_manager.cc:4841] TS 54ecd351144b4e969a4d723c7ff023d6 (127.12.138.195:32989): tablet 2dce9862b85b401796bf57840306ccac (table test-table [id=f98b5aedc3224e7383888385628b48a8]) successfully deleted
W20260811 02:56:11.766866 2765 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.12.138.193:34809 (error 108)
I20260811 02:56:11.767138 12843 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDawEwS/build/debug/bin/kudu with pid 2994
I20260811 02:56:11.769335 3315 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:56:11.768921 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.138.193:34809 (local address 127.0.0.1:36650)
0811 02:56:11.769122 (+ 201us) negotiation.cc:107] Waiting for socket to connect
0811 02:56:11.769240 (+ 118us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.138.193:34809: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":121}
W20260811 02:56:11.769577 2765 catalog_manager.cc:4584] TS 566fc2a50bfa404ebfa81c06ec9a85c9 (127.12.138.193:34809): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 2dce9862b85b401796bf57840306ccac: Network error: Client connection negotiation failed: client connection to 127.12.138.193:34809: connect: Connection refused (error 111)
I20260811 02:56:11.776813 12843 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDawEwS/build/debug/bin/kudu with pid 3143
I20260811 02:56:11.785650 12843 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDawEwS/build/debug/bin/kudu with pid 2745
2026-08-11T02:56:11Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/4 (2460 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/5
2026-08-11T02:56:11Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T02:56:11Z Disabled control of system clock
I20260811 02:56:11.820451 12843 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDawEwS/build/debug/bin/kudu
/tmp/dist-test-taskDawEwS/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786416707705494-12843-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786416707705494-12843-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786416707705494-12843-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786416707705494-12843-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.138.254:43885
--webserver_interface=127.12.138.254
--webserver_port=0
--builtin_ntp_servers=127.12.138.212:35313
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.12.138.254:43885
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260811 02:56:11.938318 3350 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 02:56:11.938609 3350 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 02:56:11.938647 3350 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 02:56:11.943634 3350 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 02:56:11.943710 3350 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 02:56:11.943727 3350 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 02:56:11.943744 3350 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 02:56:11.943760 3350 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 02:56:11.948731 3350 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.138.212:35313
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786416707705494-12843-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786416707705494-12843-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.12.138.254:43885
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.138.254:43885
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786416707705494-12843-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.12.138.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3350
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786416707705494-12843-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 02:09:49 UTC on ccycloud.quasar-fsnqqc.root.comops.site
build id 4579570
I20260811 02:56:11.949908 3350 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 02:56:11.951256 3350 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 02:56:11.959925 3359 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:56:11.959976 3356 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:56:11.959966 3357 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:56:11.960114 3350 server_base.cc:1034] running on GCE node
I20260811 02:56:11.960839 3350 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 02:56:11.962033 3350 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 02:56:11.963196 3350 hybrid_clock.cc:648] HybridClock initialized: now 1786416971963183 us; error 28 us; skew 500 ppm
I20260811 02:56:11.965087 3350 webserver.cc:460] Webserver started at http://127.12.138.254:40431/ using document root <none> and password file <none>
I20260811 02:56:11.965767 3350 fs_manager.cc:359] Metadata directory not provided
I20260811 02:56:11.965837 3350 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 02:56:11.966125 3350 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 02:56:11.968331 3350 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786416707705494-12843-0/minicluster-data/master-0/data/instance:
uuid: "8b5b4e318d3a43849e86e88ba9d867e8"
format_stamp: "Formatted at 2026-08-11 02:56:11 on dist-test-slave-9gcw"
server_key: "1a1ebc6782fe76a1bf7dba6c74baa336"
server_key_iv: "4abe099bc6b1b987889d54bff41fa313"
server_key_version: "encryptionkey@0"
I20260811 02:56:11.968962 3350 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786416707705494-12843-0/minicluster-data/master-0/wal/instance:
uuid: "8b5b4e318d3a43849e86e88ba9d867e8"
format_stamp: "Formatted at 2026-08-11 02:56:11 on dist-test-slave-9gcw"
server_key: "1a1ebc6782fe76a1bf7dba6c74baa336"
server_key_iv: "4abe099bc6b1b987889d54bff41fa313"
server_key_version: "encryptionkey@0"
I20260811 02:56:11.973011 3350 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.000s
I20260811 02:56:11.975633 3365 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 02:56:11.976998 3350 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260811 02:56:11.977178 3350 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786416707705494-12843-0/minicluster-data/master-0/data,/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786416707705494-12843-0/minicluster-data/master-0/wal
uuid: "8b5b4e318d3a43849e86e88ba9d867e8"
format_stamp: "Formatted at 2026-08-11 02:56:11 on dist-test-slave-9gcw"
server_key: "1a1ebc6782fe76a1bf7dba6c74baa336"
server_key_iv: "4abe099bc6b1b987889d54bff41fa313"
server_key_version: "encryptionkey@0"
I20260811 02:56:11.977320 3350 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786416707705494-12843-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786416707705494-12843-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786416707705494-12843-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 02:56:12.017220 3350 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 02:56:12.018007 3350 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 02:56:12.018242 3350 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 02:56:12.025386 3350 rpc_server.cc:307] RPC server started. Bound to: 127.12.138.254:43885
I20260811 02:56:12.025416 3423 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.138.254:43885 every 8 connection(s)
I20260811 02:56:12.026369 3350 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786416707705494-12843-0/minicluster-data/master-0/data/info.pb
I20260811 02:56:12.027299 12843 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDawEwS/build/debug/bin/kudu as pid 3350
I20260811 02:56:12.027446 12843 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786416707705494-12843-0/minicluster-data/master-0/wal/instance
I20260811 02:56:12.027834 12843 external_mini_cluster.cc:1428] Setting key 3034964da8d45c8b955790465e90891c
I20260811 02:56:12.030385 3424 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 02:56:12.036404 3424 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8b5b4e318d3a43849e86e88ba9d867e8: Bootstrap starting.
I20260811 02:56:12.037786 3427 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:56:12.029377 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40124 (local address 127.12.138.254:43885)
0811 02:56:12.030066 (+ 689us) server_negotiation.cc:206] Beginning negotiation
0811 02:56:12.030083 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0811 02:56:12.030122 (+ 39us) server_negotiation.cc:407] Connection header received
0811 02:56:12.030884 (+ 762us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:56:12.030904 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:56:12.031348 (+ 444us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:56:12.031735 (+ 387us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:56:12.032189 (+ 454us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:56:12.033249 (+ 1060us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:56:12.034162 (+ 913us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:56:12.034665 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:56:12.034842 (+ 177us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:56:12.035853 (+ 1011us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:56:12.035858 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:56:12.035871 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 02:56:12.036292 (+ 421us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:56:12.036354 (+ 62us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 02:56:12.036376 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:56:12.036491 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0811 02:56:12.036647 (+ 156us) server_negotiation.cc:299] Negotiation successful
0811 02:56:12.036941 (+ 294us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":317,"thread_start_us":155,"threads_started":1}
I20260811 02:56:12.039072 3424 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8b5b4e318d3a43849e86e88ba9d867e8: Neither blocks nor log segments found. Creating new log.
I20260811 02:56:12.040017 3424 log.cc:824] T 00000000000000000000000000000000 P 8b5b4e318d3a43849e86e88ba9d867e8: Log is configured to *not* fsync() on all Append() calls
I20260811 02:56:12.042177 3424 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8b5b4e318d3a43849e86e88ba9d867e8: No bootstrap required, opened a new log
I20260811 02:56:12.046321 3424 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8b5b4e318d3a43849e86e88ba9d867e8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8b5b4e318d3a43849e86e88ba9d867e8" member_type: VOTER last_known_addr { host: "127.12.138.254" port: 43885 } }
I20260811 02:56:12.046629 3424 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8b5b4e318d3a43849e86e88ba9d867e8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 02:56:12.046712 3424 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8b5b4e318d3a43849e86e88ba9d867e8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8b5b4e318d3a43849e86e88ba9d867e8, State: Initialized, Role: FOLLOWER
I20260811 02:56:12.047173 3424 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8b5b4e318d3a43849e86e88ba9d867e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8b5b4e318d3a43849e86e88ba9d867e8" member_type: VOTER last_known_addr { host: "127.12.138.254" port: 43885 } }
I20260811 02:56:12.047329 3424 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8b5b4e318d3a43849e86e88ba9d867e8 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 02:56:12.047397 3424 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8b5b4e318d3a43849e86e88ba9d867e8 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 02:56:12.047514 3424 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8b5b4e318d3a43849e86e88ba9d867e8 [term 0 FOLLOWER]: Advancing to term 1
I20260811 02:56:12.048497 3424 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8b5b4e318d3a43849e86e88ba9d867e8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8b5b4e318d3a43849e86e88ba9d867e8" member_type: VOTER last_known_addr { host: "127.12.138.254" port: 43885 } }
I20260811 02:56:12.048847 3424 leader_election.cc:302] T 00000000000000000000000000000000 P 8b5b4e318d3a43849e86e88ba9d867e8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8b5b4e318d3a43849e86e88ba9d867e8; no voters:
I20260811 02:56:12.049278 3424 leader_election.cc:288] T 00000000000000000000000000000000 P 8b5b4e318d3a43849e86e88ba9d867e8 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 02:56:12.049364 3429 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8b5b4e318d3a43849e86e88ba9d867e8 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 02:56:12.049643 3429 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8b5b4e318d3a43849e86e88ba9d867e8 [term 1 LEADER]: Becoming Leader. State: Replica: 8b5b4e318d3a43849e86e88ba9d867e8, State: Running, Role: LEADER
I20260811 02:56:12.049989 3429 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8b5b4e318d3a43849e86e88ba9d867e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8b5b4e318d3a43849e86e88ba9d867e8" member_type: VOTER last_known_addr { host: "127.12.138.254" port: 43885 } }
I20260811 02:56:12.050577 3424 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8b5b4e318d3a43849e86e88ba9d867e8 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 02:56:12.051821 3430 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8b5b4e318d3a43849e86e88ba9d867e8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8b5b4e318d3a43849e86e88ba9d867e8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8b5b4e318d3a43849e86e88ba9d867e8" member_type: VOTER last_known_addr { host: "127.12.138.254" port: 43885 } } }
I20260811 02:56:12.051836 3431 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8b5b4e318d3a43849e86e88ba9d867e8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8b5b4e318d3a43849e86e88ba9d867e8. Latest consensus state: current_term: 1 leader_uuid: "8b5b4e318d3a43849e86e88ba9d867e8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8b5b4e318d3a43849e86e88ba9d867e8" member_type: VOTER last_known_addr { host: "127.12.138.254" port: 43885 } } }
I20260811 02:56:12.052029 3431 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8b5b4e318d3a43849e86e88ba9d867e8 [sys.catalog]: This master's current role is: LEADER
I20260811 02:56:12.052353 3430 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8b5b4e318d3a43849e86e88ba9d867e8 [sys.catalog]: This master's current role is: LEADER
I20260811 02:56:12.052472 3440 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 02:56:12.055943 3440 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 02:56:12.061940 3440 catalog_manager.cc:1347] Generated new cluster ID: 54966a82ec3a420c9ee53e6b65792e54
I20260811 02:56:12.062019 3440 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 02:56:12.077591 3440 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 02:56:12.078761 3440 catalog_manager.cc:1504] Loading token signing keys...
I20260811 02:56:12.093652 3440 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8b5b4e318d3a43849e86e88ba9d867e8: Generated new TSK 0
I20260811 02:56:12.094458 3440 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 02:56:12.098172 12843 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDawEwS/build/debug/bin/kudu
/tmp/dist-test-taskDawEwS/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786416707705494-12843-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786416707705494-12843-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786416707705494-12843-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786416707705494-12843-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.138.193:0
--local_ip_for_outbound_sockets=127.12.138.193
--webserver_interface=127.12.138.193
--webserver_port=0
--tserver_master_addrs=127.12.138.254:43885
--builtin_ntp_servers=127.12.138.212:35313
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260811 02:56:12.222208 3451 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 02:56:12.222529 3451 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 02:56:12.222605 3451 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 02:56:12.226663 3451 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 02:56:12.226740 3451 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 02:56:12.226835 3451 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.138.193
I20260811 02:56:12.231648 3451 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.138.212:35313
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786416707705494-12843-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786416707705494-12843-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.138.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786416707705494-12843-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.12.138.193
--webserver_port=0
--tserver_master_addrs=127.12.138.254:43885
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3451
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.138.193
--log_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786416707705494-12843-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 02:09:49 UTC on ccycloud.quasar-fsnqqc.root.comops.site
build id 4579570
I20260811 02:56:12.232828 3451 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 02:56:12.234356 3451 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 02:56:12.244542 3458 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:56:12.244542 3460 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:56:12.244536 3457 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:56:12.244845 3451 server_base.cc:1034] running on GCE node
I20260811 02:56:12.245334 3451 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 02:56:12.245947 3451 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 02:56:12.247113 3451 hybrid_clock.cc:648] HybridClock initialized: now 1786416972247093 us; error 32 us; skew 500 ppm
I20260811 02:56:12.248884 3451 webserver.cc:460] Webserver started at http://127.12.138.193:33923/ using document root <none> and password file <none>
I20260811 02:56:12.249492 3451 fs_manager.cc:359] Metadata directory not provided
I20260811 02:56:12.249558 3451 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 02:56:12.249730 3451 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 02:56:12.251842 3451 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786416707705494-12843-0/minicluster-data/ts-0/data/instance:
uuid: "ebd8dfc93d8a4aa998fff3349dd5c8d5"
format_stamp: "Formatted at 2026-08-11 02:56:12 on dist-test-slave-9gcw"
server_key: "4edac7cd47f4e9d1c2ff3a846fb09ffd"
server_key_iv: "8c799a0e597020f950b0b3717c80d356"
server_key_version: "encryptionkey@0"
I20260811 02:56:12.252385 3451 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786416707705494-12843-0/minicluster-data/ts-0/wal/instance:
uuid: "ebd8dfc93d8a4aa998fff3349dd5c8d5"
format_stamp: "Formatted at 2026-08-11 02:56:12 on dist-test-slave-9gcw"
server_key: "4edac7cd47f4e9d1c2ff3a846fb09ffd"
server_key_iv: "8c799a0e597020f950b0b3717c80d356"
server_key_version: "encryptionkey@0"
I20260811 02:56:12.256093 3451 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.002s
I20260811 02:56:12.258628 3467 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 02:56:12.259938 3451 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260811 02:56:12.260043 3451 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786416707705494-12843-0/minicluster-data/ts-0/data,/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786416707705494-12843-0/minicluster-data/ts-0/wal
uuid: "ebd8dfc93d8a4aa998fff3349dd5c8d5"
format_stamp: "Formatted at 2026-08-11 02:56:12 on dist-test-slave-9gcw"
server_key: "4edac7cd47f4e9d1c2ff3a846fb09ffd"
server_key_iv: "8c799a0e597020f950b0b3717c80d356"
server_key_version: "encryptionkey@0"
I20260811 02:56:12.260125 3451 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786416707705494-12843-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786416707705494-12843-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786416707705494-12843-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 02:56:12.279501 3451 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 02:56:12.280617 3451 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 02:56:12.280864 3451 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 02:56:12.281772 3451 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 02:56:12.283191 3451 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 02:56:12.283280 3451 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 02:56:12.283337 3451 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 02:56:12.283367 3451 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 02:56:12.296052 3451 rpc_server.cc:307] RPC server started. Bound to: 127.12.138.193:36885
I20260811 02:56:12.296072 3591 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.138.193:36885 every 8 connection(s)
I20260811 02:56:12.297053 3451 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786416707705494-12843-0/minicluster-data/ts-0/data/info.pb
I20260811 02:56:12.306459 12843 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDawEwS/build/debug/bin/kudu as pid 3451
I20260811 02:56:12.306622 12843 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786416707705494-12843-0/minicluster-data/ts-0/wal/instance
I20260811 02:56:12.306744 3427 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:56:12.299059 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.138.193:49389 (local address 127.12.138.254:43885)
0811 02:56:12.299199 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0811 02:56:12.299203 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 02:56:12.300296 (+ 1093us) server_negotiation.cc:407] Connection header received
0811 02:56:12.301810 (+ 1514us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:56:12.301813 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:56:12.301866 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:56:12.301934 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:56:12.302824 (+ 890us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:56:12.303389 (+ 565us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:56:12.304599 (+ 1210us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:56:12.304839 (+ 240us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:56:12.304953 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:56:12.306196 (+ 1243us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:56:12.306199 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:56:12.306202 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 02:56:12.306215 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:56:12.306243 (+ 28us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 02:56:12.306252 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:56:12.306365 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0811 02:56:12.306497 (+ 132us) server_negotiation.cc:299] Negotiation successful
0811 02:56:12.306588 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260811 02:56:12.307018 12843 external_mini_cluster.cc:1428] Setting key 64f0ede76ddec3fbe8d510ae459ab5d7
I20260811 02:56:12.307677 3594 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:56:12.299469 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.138.254:43885 (local address 127.12.138.193:49389)
0811 02:56:12.300148 (+ 679us) negotiation.cc:107] Waiting for socket to connect
0811 02:56:12.300191 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0811 02:56:12.301580 (+ 1389us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:56:12.302077 (+ 497us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:56:12.302083 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:56:12.302169 (+ 86us) client_negotiation.cc:189] Negotiated authn=SASL
0811 02:56:12.302669 (+ 500us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:56:12.302682 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:56:12.303543 (+ 861us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:56:12.303546 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:56:12.304457 (+ 911us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:56:12.304467 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:56:12.305000 (+ 533us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:56:12.305003 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:56:12.305237 (+ 234us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:56:12.305920 (+ 683us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 02:56:12.305921 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0811 02:56:12.305964 (+ 43us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 02:56:12.306032 (+ 68us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 02:56:12.306332 (+ 300us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 02:56:12.306343 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 02:56:12.306351 (+ 8us) client_negotiation.cc:769] Sending connection context
0811 02:56:12.306497 (+ 146us) client_negotiation.cc:240] Negotiation successful
0811 02:56:12.306787 (+ 290us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":266,"thread_start_us":131,"threads_started":1}
I20260811 02:56:12.308990 3592 heartbeater.cc:347] Connected to a master server at 127.12.138.254:43885
I20260811 02:56:12.309237 12843 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDawEwS/build/debug/bin/kudu
/tmp/dist-test-taskDawEwS/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786416707705494-12843-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786416707705494-12843-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786416707705494-12843-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786416707705494-12843-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.138.194:0
--local_ip_for_outbound_sockets=127.12.138.194
--webserver_interface=127.12.138.194
--webserver_port=0
--tserver_master_addrs=127.12.138.254:43885
--builtin_ntp_servers=127.12.138.212:35313
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260811 02:56:12.309283 3592 heartbeater.cc:464] Registering TS with master...
I20260811 02:56:12.309924 3592 heartbeater.cc:511] Master 127.12.138.254:43885 requested a full tablet report, sending...
I20260811 02:56:12.311635 3382 ts_manager.cc:194] Registered new tserver with Master: ebd8dfc93d8a4aa998fff3349dd5c8d5 (127.12.138.193:36885)
I20260811 02:56:12.313315 3382 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.12.138.193:49389
W20260811 02:56:12.426343 3595 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 02:56:12.426654 3595 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 02:56:12.426700 3595 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 02:56:12.431923 3595 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 02:56:12.432011 3595 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 02:56:12.432081 3595 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.138.194
I20260811 02:56:12.437444 3595 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.138.212:35313
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786416707705494-12843-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786416707705494-12843-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.138.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786416707705494-12843-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.12.138.194
--webserver_port=0
--tserver_master_addrs=127.12.138.254:43885
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3595
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.138.194
--log_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786416707705494-12843-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 02:09:49 UTC on ccycloud.quasar-fsnqqc.root.comops.site
build id 4579570
I20260811 02:56:12.438649 3595 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 02:56:12.440127 3595 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 02:56:12.451205 3604 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:56:12.451205 3602 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:56:12.451279 3601 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:56:12.451618 3595 server_base.cc:1034] running on GCE node
I20260811 02:56:12.452133 3595 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 02:56:12.452878 3595 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 02:56:12.454061 3595 hybrid_clock.cc:648] HybridClock initialized: now 1786416972454013 us; error 70 us; skew 500 ppm
I20260811 02:56:12.456067 3595 webserver.cc:460] Webserver started at http://127.12.138.194:38657/ using document root <none> and password file <none>
I20260811 02:56:12.456687 3595 fs_manager.cc:359] Metadata directory not provided
I20260811 02:56:12.456789 3595 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 02:56:12.457090 3595 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 02:56:12.459321 3595 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786416707705494-12843-0/minicluster-data/ts-1/data/instance:
uuid: "5b9aa0c640264b13b9de4aaec74004e4"
format_stamp: "Formatted at 2026-08-11 02:56:12 on dist-test-slave-9gcw"
server_key: "76f5474b131472adfbee6addad604cb0"
server_key_iv: "d63680dcfde1ee3c78b781dbedf2f1bb"
server_key_version: "encryptionkey@0"
I20260811 02:56:12.459951 3595 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786416707705494-12843-0/minicluster-data/ts-1/wal/instance:
uuid: "5b9aa0c640264b13b9de4aaec74004e4"
format_stamp: "Formatted at 2026-08-11 02:56:12 on dist-test-slave-9gcw"
server_key: "76f5474b131472adfbee6addad604cb0"
server_key_iv: "d63680dcfde1ee3c78b781dbedf2f1bb"
server_key_version: "encryptionkey@0"
I20260811 02:56:12.464037 3595 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.004s
I20260811 02:56:12.466792 3614 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 02:56:12.468075 3595 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260811 02:56:12.468223 3595 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786416707705494-12843-0/minicluster-data/ts-1/data,/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786416707705494-12843-0/minicluster-data/ts-1/wal
uuid: "5b9aa0c640264b13b9de4aaec74004e4"
format_stamp: "Formatted at 2026-08-11 02:56:12 on dist-test-slave-9gcw"
server_key: "76f5474b131472adfbee6addad604cb0"
server_key_iv: "d63680dcfde1ee3c78b781dbedf2f1bb"
server_key_version: "encryptionkey@0"
I20260811 02:56:12.468376 3595 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786416707705494-12843-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786416707705494-12843-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786416707705494-12843-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 02:56:12.485276 3595 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 02:56:12.486040 3595 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 02:56:12.486263 3595 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 02:56:12.486896 3595 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 02:56:12.487968 3595 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 02:56:12.488037 3595 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 02:56:12.488106 3595 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 02:56:12.488155 3595 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 02:56:12.497567 3595 rpc_server.cc:307] RPC server started. Bound to: 127.12.138.194:42409
I20260811 02:56:12.497611 3744 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.138.194:42409 every 8 connection(s)
I20260811 02:56:12.498553 3595 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786416707705494-12843-0/minicluster-data/ts-1/data/info.pb
I20260811 02:56:12.508085 3427 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:56:12.500450 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.138.194:44779 (local address 127.12.138.254:43885)
0811 02:56:12.500606 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0811 02:56:12.500610 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 02:56:12.501707 (+ 1097us) server_negotiation.cc:407] Connection header received
0811 02:56:12.503162 (+ 1455us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:56:12.503165 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:56:12.503217 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:56:12.503334 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:56:12.504265 (+ 931us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:56:12.504792 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:56:12.505981 (+ 1189us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:56:12.506195 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:56:12.506288 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:56:12.507474 (+ 1186us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:56:12.507477 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:56:12.507479 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 02:56:12.507492 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:56:12.507511 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 02:56:12.507520 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:56:12.507579 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0811 02:56:12.507905 (+ 326us) server_negotiation.cc:299] Negotiation successful
0811 02:56:12.507963 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260811 02:56:12.508076 12843 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDawEwS/build/debug/bin/kudu as pid 3595
I20260811 02:56:12.508249 12843 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786416707705494-12843-0/minicluster-data/ts-1/wal/instance
I20260811 02:56:12.508616 12843 external_mini_cluster.cc:1428] Setting key 5cdf6d61393e5887d1c440f7874a669a
I20260811 02:56:12.509387 3748 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:56:12.500868 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.138.254:43885 (local address 127.12.138.194:44779)
0811 02:56:12.501559 (+ 691us) negotiation.cc:107] Waiting for socket to connect
0811 02:56:12.501602 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0811 02:56:12.502927 (+ 1325us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:56:12.503511 (+ 584us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:56:12.503519 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:56:12.503604 (+ 85us) client_negotiation.cc:189] Negotiated authn=SASL
0811 02:56:12.504104 (+ 500us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:56:12.504115 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:56:12.504903 (+ 788us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:56:12.504905 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:56:12.505867 (+ 962us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:56:12.505876 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:56:12.506375 (+ 499us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:56:12.506379 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:56:12.506547 (+ 168us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:56:12.507206 (+ 659us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 02:56:12.507208 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0811 02:56:12.507251 (+ 43us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 02:56:12.507321 (+ 70us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 02:56:12.507624 (+ 303us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 02:56:12.507634 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 02:56:12.507737 (+ 103us) client_negotiation.cc:769] Sending connection context
0811 02:56:12.507909 (+ 172us) client_negotiation.cc:240] Negotiation successful
0811 02:56:12.508275 (+ 366us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":273,"thread_start_us":113,"threads_started":1}
I20260811 02:56:12.510699 3746 heartbeater.cc:347] Connected to a master server at 127.12.138.254:43885
I20260811 02:56:12.510964 3746 heartbeater.cc:464] Registering TS with master...
I20260811 02:56:12.510954 12843 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDawEwS/build/debug/bin/kudu
/tmp/dist-test-taskDawEwS/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786416707705494-12843-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786416707705494-12843-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786416707705494-12843-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786416707705494-12843-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.138.195:0
--local_ip_for_outbound_sockets=127.12.138.195
--webserver_interface=127.12.138.195
--webserver_port=0
--tserver_master_addrs=127.12.138.254:43885
--builtin_ntp_servers=127.12.138.212:35313
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260811 02:56:12.511672 3746 heartbeater.cc:511] Master 127.12.138.254:43885 requested a full tablet report, sending...
I20260811 02:56:12.512987 3382 ts_manager.cc:194] Registered new tserver with Master: 5b9aa0c640264b13b9de4aaec74004e4 (127.12.138.194:42409)
I20260811 02:56:12.513619 3382 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.12.138.194:44779
W20260811 02:56:12.629349 3749 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 02:56:12.629686 3749 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 02:56:12.629765 3749 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 02:56:12.633913 3749 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 02:56:12.634011 3749 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 02:56:12.634116 3749 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.138.195
I20260811 02:56:12.639195 3749 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.138.212:35313
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786416707705494-12843-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786416707705494-12843-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.138.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786416707705494-12843-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.12.138.195
--webserver_port=0
--tserver_master_addrs=127.12.138.254:43885
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3749
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.138.195
--log_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786416707705494-12843-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 02:09:49 UTC on ccycloud.quasar-fsnqqc.root.comops.site
build id 4579570
I20260811 02:56:12.640357 3749 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 02:56:12.641969 3749 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 02:56:12.652282 3757 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:56:12.652320 3749 server_base.cc:1034] running on GCE node
W20260811 02:56:12.652275 3758 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:56:12.652288 3760 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:56:12.652948 3749 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 02:56:12.653662 3749 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 02:56:12.654834 3749 hybrid_clock.cc:648] HybridClock initialized: now 1786416972654800 us; error 47 us; skew 500 ppm
I20260811 02:56:12.656690 3749 webserver.cc:460] Webserver started at http://127.12.138.195:32933/ using document root <none> and password file <none>
I20260811 02:56:12.657339 3749 fs_manager.cc:359] Metadata directory not provided
I20260811 02:56:12.657407 3749 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 02:56:12.657616 3749 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 02:56:12.659678 3749 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786416707705494-12843-0/minicluster-data/ts-2/data/instance:
uuid: "b0636fd10ece4f0ab0ac6b42437d4104"
format_stamp: "Formatted at 2026-08-11 02:56:12 on dist-test-slave-9gcw"
server_key: "3432dffc09ad4db69de09eb97da72323"
server_key_iv: "f2465d395f3baf3b86c201898eda622a"
server_key_version: "encryptionkey@0"
I20260811 02:56:12.660248 3749 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786416707705494-12843-0/minicluster-data/ts-2/wal/instance:
uuid: "b0636fd10ece4f0ab0ac6b42437d4104"
format_stamp: "Formatted at 2026-08-11 02:56:12 on dist-test-slave-9gcw"
server_key: "3432dffc09ad4db69de09eb97da72323"
server_key_iv: "f2465d395f3baf3b86c201898eda622a"
server_key_version: "encryptionkey@0"
I20260811 02:56:12.664049 3749 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.000s sys 0.006s
I20260811 02:56:12.666661 3769 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 02:56:12.667901 3749 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260811 02:56:12.668018 3749 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786416707705494-12843-0/minicluster-data/ts-2/data,/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786416707705494-12843-0/minicluster-data/ts-2/wal
uuid: "b0636fd10ece4f0ab0ac6b42437d4104"
format_stamp: "Formatted at 2026-08-11 02:56:12 on dist-test-slave-9gcw"
server_key: "3432dffc09ad4db69de09eb97da72323"
server_key_iv: "f2465d395f3baf3b86c201898eda622a"
server_key_version: "encryptionkey@0"
I20260811 02:56:12.668121 3749 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786416707705494-12843-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786416707705494-12843-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786416707705494-12843-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 02:56:12.682633 3749 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 02:56:12.683284 3749 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 02:56:12.683486 3749 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 02:56:12.684056 3749 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 02:56:12.684988 3749 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 02:56:12.685057 3749 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 02:56:12.685120 3749 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 02:56:12.685180 3749 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 02:56:12.693794 3749 rpc_server.cc:307] RPC server started. Bound to: 127.12.138.195:35849
I20260811 02:56:12.694774 3749 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786416707705494-12843-0/minicluster-data/ts-2/data/info.pb
I20260811 02:56:12.695205 3902 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.138.195:35849 every 8 connection(s)
I20260811 02:56:12.699347 12843 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDawEwS/build/debug/bin/kudu as pid 3749
I20260811 02:56:12.699470 12843 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786416707705494-12843-0/minicluster-data/ts-2/wal/instance
I20260811 02:56:12.699813 12843 external_mini_cluster.cc:1428] Setting key 1e18f5d62387679cb7cab493578d0909
I20260811 02:56:12.704809 3427 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:56:12.696832 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.138.195:44827 (local address 127.12.138.254:43885)
0811 02:56:12.696976 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0811 02:56:12.696980 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 02:56:12.698130 (+ 1150us) server_negotiation.cc:407] Connection header received
0811 02:56:12.699722 (+ 1592us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:56:12.699725 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:56:12.699774 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:56:12.699849 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:56:12.700992 (+ 1143us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:56:12.701806 (+ 814us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:56:12.703008 (+ 1202us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:56:12.703222 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:56:12.703312 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:56:12.704386 (+ 1074us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:56:12.704389 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:56:12.704391 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 02:56:12.704401 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:56:12.704417 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 02:56:12.704425 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:56:12.704484 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0811 02:56:12.704620 (+ 136us) server_negotiation.cc:299] Negotiation successful
0811 02:56:12.704666 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260811 02:56:12.705670 3905 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:56:12.697281 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.138.254:43885 (local address 127.12.138.195:44827)
0811 02:56:12.697989 (+ 708us) negotiation.cc:107] Waiting for socket to connect
0811 02:56:12.698030 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0811 02:56:12.699490 (+ 1460us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:56:12.700033 (+ 543us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:56:12.700042 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:56:12.700156 (+ 114us) client_negotiation.cc:189] Negotiated authn=SASL
0811 02:56:12.700829 (+ 673us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:56:12.700849 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:56:12.701946 (+ 1097us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:56:12.701948 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:56:12.702886 (+ 938us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:56:12.702896 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:56:12.703361 (+ 465us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:56:12.703363 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:56:12.703502 (+ 139us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:56:12.704152 (+ 650us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 02:56:12.704154 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0811 02:56:12.704194 (+ 40us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 02:56:12.704256 (+ 62us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 02:56:12.704502 (+ 246us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 02:56:12.704508 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 02:56:12.704513 (+ 5us) client_negotiation.cc:769] Sending connection context
0811 02:56:12.704606 (+ 93us) client_negotiation.cc:240] Negotiation successful
0811 02:56:12.704849 (+ 243us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":260,"thread_start_us":115,"threads_started":1}
I20260811 02:56:12.706776 3903 heartbeater.cc:347] Connected to a master server at 127.12.138.254:43885
I20260811 02:56:12.707027 3903 heartbeater.cc:464] Registering TS with master...
I20260811 02:56:12.707533 3903 heartbeater.cc:511] Master 127.12.138.254:43885 requested a full tablet report, sending...
I20260811 02:56:12.708535 3382 ts_manager.cc:194] Registered new tserver with Master: b0636fd10ece4f0ab0ac6b42437d4104 (127.12.138.195:35849)
I20260811 02:56:12.709215 3382 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.12.138.195:44827
I20260811 02:56:12.713889 12843 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 02:56:12.720149 3427 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:56:12.715948 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48634 (local address 127.12.138.254:43885)
0811 02:56:12.716051 (+ 103us) server_negotiation.cc:206] Beginning negotiation
0811 02:56:12.716054 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 02:56:12.716324 (+ 270us) server_negotiation.cc:407] Connection header received
0811 02:56:12.716485 (+ 161us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:56:12.716494 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:56:12.716537 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:56:12.716625 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:56:12.716924 (+ 299us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:56:12.717535 (+ 611us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:56:12.718464 (+ 929us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:56:12.718698 (+ 234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:56:12.718810 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:56:12.719709 (+ 899us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:56:12.719713 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:56:12.719716 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 02:56:12.719736 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:56:12.719754 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 02:56:12.719762 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:56:12.719846 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0811 02:56:12.719946 (+ 100us) server_negotiation.cc:299] Negotiation successful
0811 02:56:12.720000 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260811 02:56:12.723163 3382 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:48634:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 02:56:12.726385 3382 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 02:56:12.741008 3915 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:56:12.735128 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37956 (local address 127.12.138.194:42409)
0811 02:56:12.735549 (+ 421us) server_negotiation.cc:206] Beginning negotiation
0811 02:56:12.735566 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0811 02:56:12.735598 (+ 32us) server_negotiation.cc:407] Connection header received
0811 02:56:12.735690 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:56:12.735699 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:56:12.735953 (+ 254us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:56:12.736151 (+ 198us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 02:56:12.736749 (+ 598us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:56:12.737867 (+ 1118us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:56:12.739618 (+ 1751us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:56:12.740249 (+ 631us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:56:12.740393 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:56:12.740594 (+ 201us) server_negotiation.cc:1035] Waiting for connection context
0811 02:56:12.740722 (+ 128us) server_negotiation.cc:299] Negotiation successful
0811 02:56:12.740831 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":298,"thread_start_us":194,"threads_started":1}
I20260811 02:56:12.741003 3914 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:56:12.734971 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.138.194:42409 (local address 127.0.0.1:37956)
0811 02:56:12.735328 (+ 357us) negotiation.cc:107] Waiting for socket to connect
0811 02:56:12.735389 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0811 02:56:12.735525 (+ 136us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:56:12.736384 (+ 859us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:56:12.736391 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:56:12.736418 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 02:56:12.736586 (+ 168us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:56:12.736595 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:56:12.737984 (+ 1389us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:56:12.737987 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:56:12.739496 (+ 1509us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:56:12.739506 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:56:12.740390 (+ 884us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:56:12.740394 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:56:12.740531 (+ 137us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:56:12.740573 (+ 42us) client_negotiation.cc:769] Sending connection context
0811 02:56:12.740701 (+ 128us) client_negotiation.cc:240] Negotiation successful
0811 02:56:12.740831 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":171,"mutex_wait_us":36,"thread_start_us":103,"threads_started":1}
I20260811 02:56:12.742660 3916 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:56:12.735816 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.138.193:36885 (local address 127.0.0.1:54104)
0811 02:56:12.736451 (+ 635us) negotiation.cc:107] Waiting for socket to connect
0811 02:56:12.736467 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 02:56:12.736569 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:56:12.737492 (+ 923us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:56:12.737495 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:56:12.737516 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 02:56:12.737620 (+ 104us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:56:12.737626 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:56:12.739404 (+ 1778us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:56:12.739408 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:56:12.740727 (+ 1319us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:56:12.740738 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:56:12.742171 (+ 1433us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:56:12.742175 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:56:12.742297 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:56:12.742334 (+ 37us) client_negotiation.cc:769] Sending connection context
0811 02:56:12.742397 (+ 63us) client_negotiation.cc:240] Negotiation successful
0811 02:56:12.742501 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":559,"mutex_wait_us":85,"thread_start_us":107,"threads_started":1}
I20260811 02:56:12.742704 3919 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:56:12.736609 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45752 (local address 127.12.138.195:35849)
0811 02:56:12.737031 (+ 422us) server_negotiation.cc:206] Beginning negotiation
0811 02:56:12.737043 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 02:56:12.737066 (+ 23us) server_negotiation.cc:407] Connection header received
0811 02:56:12.737134 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:56:12.737139 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:56:12.737333 (+ 194us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:56:12.737503 (+ 170us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 02:56:12.737973 (+ 470us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:56:12.738974 (+ 1001us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:56:12.741012 (+ 2038us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:56:12.742054 (+ 1042us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:56:12.742186 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:56:12.742368 (+ 182us) server_negotiation.cc:1035] Waiting for connection context
0811 02:56:12.742439 (+ 71us) server_negotiation.cc:299] Negotiation successful
0811 02:56:12.742534 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":317,"thread_start_us":195,"threads_started":1}
I20260811 02:56:12.742664 3918 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:56:12.736153 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.138.195:35849 (local address 127.0.0.1:45752)
0811 02:56:12.736517 (+ 364us) negotiation.cc:107] Waiting for socket to connect
0811 02:56:12.736529 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 02:56:12.736606 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:56:12.737681 (+ 1075us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:56:12.737687 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:56:12.737720 (+ 33us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 02:56:12.737829 (+ 109us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:56:12.737836 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:56:12.739119 (+ 1283us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:56:12.739124 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:56:12.740892 (+ 1768us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:56:12.740898 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:56:12.742178 (+ 1280us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:56:12.742182 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:56:12.742289 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:56:12.742331 (+ 42us) client_negotiation.cc:769] Sending connection context
0811 02:56:12.742398 (+ 67us) client_negotiation.cc:240] Negotiation successful
0811 02:56:12.742515 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":278,"mutex_wait_us":38,"thread_start_us":100,"threads_started":1}
I20260811 02:56:12.743887 3917 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:56:12.736031 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54104 (local address 127.12.138.193:36885)
0811 02:56:12.736949 (+ 918us) server_negotiation.cc:206] Beginning negotiation
0811 02:56:12.736966 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0811 02:56:12.736993 (+ 27us) server_negotiation.cc:407] Connection header received
0811 02:56:12.737080 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:56:12.737086 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:56:12.737325 (+ 239us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:56:12.737503 (+ 178us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 02:56:12.737732 (+ 229us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:56:12.739278 (+ 1546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:56:12.740868 (+ 1590us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:56:12.742002 (+ 1134us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:56:12.743390 (+ 1388us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:56:12.743563 (+ 173us) server_negotiation.cc:1035] Waiting for connection context
0811 02:56:12.743613 (+ 50us) server_negotiation.cc:299] Negotiation successful
0811 02:56:12.743704 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":765,"thread_start_us":138,"threads_started":1}
I20260811 02:56:12.744261 3665 tablet_service.cc:1467] Processing CreateTablet for tablet 013b5b3a9dfb4b4285cbcfd16ceb2751 (DEFAULT_TABLE table=test-table [id=86da6fc5167b4fa0ac4e996150715549]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 02:56:12.745551 3665 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 013b5b3a9dfb4b4285cbcfd16ceb2751. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 02:56:12.746172 3824 tablet_service.cc:1467] Processing CreateTablet for tablet 013b5b3a9dfb4b4285cbcfd16ceb2751 (DEFAULT_TABLE table=test-table [id=86da6fc5167b4fa0ac4e996150715549]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 02:56:12.746397 3515 tablet_service.cc:1467] Processing CreateTablet for tablet 013b5b3a9dfb4b4285cbcfd16ceb2751 (DEFAULT_TABLE table=test-table [id=86da6fc5167b4fa0ac4e996150715549]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 02:56:12.747251 3824 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 013b5b3a9dfb4b4285cbcfd16ceb2751. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 02:56:12.747392 3515 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 013b5b3a9dfb4b4285cbcfd16ceb2751. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 02:56:12.754649 3923 tablet_bootstrap.cc:492] T 013b5b3a9dfb4b4285cbcfd16ceb2751 P b0636fd10ece4f0ab0ac6b42437d4104: Bootstrap starting.
I20260811 02:56:12.755154 3920 tablet_bootstrap.cc:492] T 013b5b3a9dfb4b4285cbcfd16ceb2751 P ebd8dfc93d8a4aa998fff3349dd5c8d5: Bootstrap starting.
I20260811 02:56:12.755539 3922 tablet_bootstrap.cc:492] T 013b5b3a9dfb4b4285cbcfd16ceb2751 P 5b9aa0c640264b13b9de4aaec74004e4: Bootstrap starting.
I20260811 02:56:12.757373 3920 tablet_bootstrap.cc:654] T 013b5b3a9dfb4b4285cbcfd16ceb2751 P ebd8dfc93d8a4aa998fff3349dd5c8d5: Neither blocks nor log segments found. Creating new log.
I20260811 02:56:12.757373 3923 tablet_bootstrap.cc:654] T 013b5b3a9dfb4b4285cbcfd16ceb2751 P b0636fd10ece4f0ab0ac6b42437d4104: Neither blocks nor log segments found. Creating new log.
I20260811 02:56:12.757567 3922 tablet_bootstrap.cc:654] T 013b5b3a9dfb4b4285cbcfd16ceb2751 P 5b9aa0c640264b13b9de4aaec74004e4: Neither blocks nor log segments found. Creating new log.
I20260811 02:56:12.758375 3922 log.cc:824] T 013b5b3a9dfb4b4285cbcfd16ceb2751 P 5b9aa0c640264b13b9de4aaec74004e4: Log is configured to *not* fsync() on all Append() calls
I20260811 02:56:12.758370 3923 log.cc:824] T 013b5b3a9dfb4b4285cbcfd16ceb2751 P b0636fd10ece4f0ab0ac6b42437d4104: Log is configured to *not* fsync() on all Append() calls
I20260811 02:56:12.758370 3920 log.cc:824] T 013b5b3a9dfb4b4285cbcfd16ceb2751 P ebd8dfc93d8a4aa998fff3349dd5c8d5: Log is configured to *not* fsync() on all Append() calls
I20260811 02:56:12.760949 3923 tablet_bootstrap.cc:492] T 013b5b3a9dfb4b4285cbcfd16ceb2751 P b0636fd10ece4f0ab0ac6b42437d4104: No bootstrap required, opened a new log
I20260811 02:56:12.760949 3922 tablet_bootstrap.cc:492] T 013b5b3a9dfb4b4285cbcfd16ceb2751 P 5b9aa0c640264b13b9de4aaec74004e4: No bootstrap required, opened a new log
I20260811 02:56:12.761236 3922 ts_tablet_manager.cc:1397] T 013b5b3a9dfb4b4285cbcfd16ceb2751 P 5b9aa0c640264b13b9de4aaec74004e4: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260811 02:56:12.761236 3923 ts_tablet_manager.cc:1397] T 013b5b3a9dfb4b4285cbcfd16ceb2751 P b0636fd10ece4f0ab0ac6b42437d4104: Time spent bootstrapping tablet: real 0.007s user 0.002s sys 0.004s
I20260811 02:56:12.761402 3920 tablet_bootstrap.cc:492] T 013b5b3a9dfb4b4285cbcfd16ceb2751 P ebd8dfc93d8a4aa998fff3349dd5c8d5: No bootstrap required, opened a new log
I20260811 02:56:12.761620 3920 ts_tablet_manager.cc:1397] T 013b5b3a9dfb4b4285cbcfd16ceb2751 P ebd8dfc93d8a4aa998fff3349dd5c8d5: Time spent bootstrapping tablet: real 0.007s user 0.001s sys 0.004s
I20260811 02:56:12.765583 3920 raft_consensus.cc:348] T 013b5b3a9dfb4b4285cbcfd16ceb2751 P ebd8dfc93d8a4aa998fff3349dd5c8d5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b9aa0c640264b13b9de4aaec74004e4" member_type: VOTER last_known_addr { host: "127.12.138.194" port: 42409 } } peers { permanent_uuid: "ebd8dfc93d8a4aa998fff3349dd5c8d5" member_type: VOTER last_known_addr { host: "127.12.138.193" port: 36885 } } peers { permanent_uuid: "b0636fd10ece4f0ab0ac6b42437d4104" member_type: VOTER last_known_addr { host: "127.12.138.195" port: 35849 } }
I20260811 02:56:12.765875 3920 raft_consensus.cc:374] T 013b5b3a9dfb4b4285cbcfd16ceb2751 P ebd8dfc93d8a4aa998fff3349dd5c8d5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 02:56:12.765939 3920 raft_consensus.cc:729] T 013b5b3a9dfb4b4285cbcfd16ceb2751 P ebd8dfc93d8a4aa998fff3349dd5c8d5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ebd8dfc93d8a4aa998fff3349dd5c8d5, State: Initialized, Role: FOLLOWER
I20260811 02:56:12.766310 3922 raft_consensus.cc:348] T 013b5b3a9dfb4b4285cbcfd16ceb2751 P 5b9aa0c640264b13b9de4aaec74004e4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b9aa0c640264b13b9de4aaec74004e4" member_type: VOTER last_known_addr { host: "127.12.138.194" port: 42409 } } peers { permanent_uuid: "ebd8dfc93d8a4aa998fff3349dd5c8d5" member_type: VOTER last_known_addr { host: "127.12.138.193" port: 36885 } } peers { permanent_uuid: "b0636fd10ece4f0ab0ac6b42437d4104" member_type: VOTER last_known_addr { host: "127.12.138.195" port: 35849 } }
I20260811 02:56:12.766310 3923 raft_consensus.cc:348] T 013b5b3a9dfb4b4285cbcfd16ceb2751 P b0636fd10ece4f0ab0ac6b42437d4104 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b9aa0c640264b13b9de4aaec74004e4" member_type: VOTER last_known_addr { host: "127.12.138.194" port: 42409 } } peers { permanent_uuid: "ebd8dfc93d8a4aa998fff3349dd5c8d5" member_type: VOTER last_known_addr { host: "127.12.138.193" port: 36885 } } peers { permanent_uuid: "b0636fd10ece4f0ab0ac6b42437d4104" member_type: VOTER last_known_addr { host: "127.12.138.195" port: 35849 } }
I20260811 02:56:12.766592 3922 raft_consensus.cc:374] T 013b5b3a9dfb4b4285cbcfd16ceb2751 P 5b9aa0c640264b13b9de4aaec74004e4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 02:56:12.766592 3923 raft_consensus.cc:374] T 013b5b3a9dfb4b4285cbcfd16ceb2751 P b0636fd10ece4f0ab0ac6b42437d4104 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 02:56:12.766649 3923 raft_consensus.cc:729] T 013b5b3a9dfb4b4285cbcfd16ceb2751 P b0636fd10ece4f0ab0ac6b42437d4104 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b0636fd10ece4f0ab0ac6b42437d4104, State: Initialized, Role: FOLLOWER
I20260811 02:56:12.766649 3922 raft_consensus.cc:729] T 013b5b3a9dfb4b4285cbcfd16ceb2751 P 5b9aa0c640264b13b9de4aaec74004e4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5b9aa0c640264b13b9de4aaec74004e4, State: Initialized, Role: FOLLOWER
I20260811 02:56:12.766731 3920 consensus_queue.cc:260] T 013b5b3a9dfb4b4285cbcfd16ceb2751 P ebd8dfc93d8a4aa998fff3349dd5c8d5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b9aa0c640264b13b9de4aaec74004e4" member_type: VOTER last_known_addr { host: "127.12.138.194" port: 42409 } } peers { permanent_uuid: "ebd8dfc93d8a4aa998fff3349dd5c8d5" member_type: VOTER last_known_addr { host: "127.12.138.193" port: 36885 } } peers { permanent_uuid: "b0636fd10ece4f0ab0ac6b42437d4104" member_type: VOTER last_known_addr { host: "127.12.138.195" port: 35849 } }
I20260811 02:56:12.767128 3922 consensus_queue.cc:260] T 013b5b3a9dfb4b4285cbcfd16ceb2751 P 5b9aa0c640264b13b9de4aaec74004e4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b9aa0c640264b13b9de4aaec74004e4" member_type: VOTER last_known_addr { host: "127.12.138.194" port: 42409 } } peers { permanent_uuid: "ebd8dfc93d8a4aa998fff3349dd5c8d5" member_type: VOTER last_known_addr { host: "127.12.138.193" port: 36885 } } peers { permanent_uuid: "b0636fd10ece4f0ab0ac6b42437d4104" member_type: VOTER last_known_addr { host: "127.12.138.195" port: 35849 } }
I20260811 02:56:12.767170 3923 consensus_queue.cc:260] T 013b5b3a9dfb4b4285cbcfd16ceb2751 P b0636fd10ece4f0ab0ac6b42437d4104 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b9aa0c640264b13b9de4aaec74004e4" member_type: VOTER last_known_addr { host: "127.12.138.194" port: 42409 } } peers { permanent_uuid: "ebd8dfc93d8a4aa998fff3349dd5c8d5" member_type: VOTER last_known_addr { host: "127.12.138.193" port: 36885 } } peers { permanent_uuid: "b0636fd10ece4f0ab0ac6b42437d4104" member_type: VOTER last_known_addr { host: "127.12.138.195" port: 35849 } }
I20260811 02:56:12.768034 3920 ts_tablet_manager.cc:1428] T 013b5b3a9dfb4b4285cbcfd16ceb2751 P ebd8dfc93d8a4aa998fff3349dd5c8d5: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260811 02:56:12.768167 3903 heartbeater.cc:503] Master 127.12.138.254:43885 was elected leader, sending a full tablet report...
I20260811 02:56:12.768209 3922 ts_tablet_manager.cc:1428] T 013b5b3a9dfb4b4285cbcfd16ceb2751 P 5b9aa0c640264b13b9de4aaec74004e4: Time spent starting tablet: real 0.007s user 0.006s sys 0.000s
I20260811 02:56:12.768209 3923 ts_tablet_manager.cc:1428] T 013b5b3a9dfb4b4285cbcfd16ceb2751 P b0636fd10ece4f0ab0ac6b42437d4104: Time spent starting tablet: real 0.007s user 0.006s sys 0.000s
I20260811 02:56:12.768381 3592 heartbeater.cc:503] Master 127.12.138.254:43885 was elected leader, sending a full tablet report...
I20260811 02:56:12.769487 3746 heartbeater.cc:503] Master 127.12.138.254:43885 was elected leader, sending a full tablet report...
W20260811 02:56:12.798453 3593 tablet.cc:2369] T 013b5b3a9dfb4b4285cbcfd16ceb2751 P ebd8dfc93d8a4aa998fff3349dd5c8d5: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 02:56:12.870034 3928 raft_consensus.cc:482] T 013b5b3a9dfb4b4285cbcfd16ceb2751 P ebd8dfc93d8a4aa998fff3349dd5c8d5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 02:56:12.870271 3928 raft_consensus.cc:504] T 013b5b3a9dfb4b4285cbcfd16ceb2751 P ebd8dfc93d8a4aa998fff3349dd5c8d5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b9aa0c640264b13b9de4aaec74004e4" member_type: VOTER last_known_addr { host: "127.12.138.194" port: 42409 } } peers { permanent_uuid: "ebd8dfc93d8a4aa998fff3349dd5c8d5" member_type: VOTER last_known_addr { host: "127.12.138.193" port: 36885 } } peers { permanent_uuid: "b0636fd10ece4f0ab0ac6b42437d4104" member_type: VOTER last_known_addr { host: "127.12.138.195" port: 35849 } }
I20260811 02:56:12.871394 3928 leader_election.cc:288] T 013b5b3a9dfb4b4285cbcfd16ceb2751 P ebd8dfc93d8a4aa998fff3349dd5c8d5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5b9aa0c640264b13b9de4aaec74004e4 (127.12.138.194:42409), b0636fd10ece4f0ab0ac6b42437d4104 (127.12.138.195:35849)
I20260811 02:56:12.877049 3915 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:56:12.871827 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.138.193:32773 (local address 127.12.138.194:42409)
0811 02:56:12.871968 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0811 02:56:12.871975 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 02:56:12.872112 (+ 137us) server_negotiation.cc:407] Connection header received
0811 02:56:12.872452 (+ 340us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:56:12.872463 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:56:12.872515 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:56:12.872616 (+ 101us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 02:56:12.872954 (+ 338us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:56:12.873940 (+ 986us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:56:12.875624 (+ 1684us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:56:12.876304 (+ 680us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:56:12.876441 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:56:12.876497 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0811 02:56:12.876821 (+ 324us) server_negotiation.cc:299] Negotiation successful
0811 02:56:12.876898 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260811 02:56:12.877049 3919 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:56:12.872277 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.138.193:43517 (local address 127.12.138.195:35849)
0811 02:56:12.872410 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0811 02:56:12.872415 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 02:56:12.872653 (+ 238us) server_negotiation.cc:407] Connection header received
0811 02:56:12.872810 (+ 157us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:56:12.872812 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:56:12.872884 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:56:12.872973 (+ 89us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 02:56:12.873274 (+ 301us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:56:12.874067 (+ 793us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:56:12.875624 (+ 1557us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:56:12.876304 (+ 680us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:56:12.876441 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:56:12.876497 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0811 02:56:12.876820 (+ 323us) server_negotiation.cc:299] Negotiation successful
0811 02:56:12.876898 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260811 02:56:12.877099 3932 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:56:12.872168 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.138.195:35849 (local address 127.12.138.193:43517)
0811 02:56:12.872522 (+ 354us) negotiation.cc:107] Waiting for socket to connect
0811 02:56:12.872549 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0811 02:56:12.872695 (+ 146us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:56:12.873000 (+ 305us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:56:12.873004 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:56:12.873024 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 02:56:12.873125 (+ 101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:56:12.873129 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:56:12.874167 (+ 1038us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:56:12.874170 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:56:12.875482 (+ 1312us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:56:12.875490 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:56:12.876442 (+ 952us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:56:12.876446 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:56:12.876636 (+ 190us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:56:12.876703 (+ 67us) client_negotiation.cc:769] Sending connection context
0811 02:56:12.876787 (+ 84us) client_negotiation.cc:240] Negotiation successful
0811 02:56:12.876898 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":286,"mutex_wait_us":162,"spinlock_wait_cycles":27264,"thread_start_us":118,"threads_started":1}
I20260811 02:56:12.877099 3931 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:56:12.871673 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.138.194:42409 (local address 127.12.138.193:32773)
0811 02:56:12.872013 (+ 340us) negotiation.cc:107] Waiting for socket to connect
0811 02:56:12.872031 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 02:56:12.872246 (+ 215us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:56:12.872649 (+ 403us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:56:12.872652 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:56:12.872693 (+ 41us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 02:56:12.872853 (+ 160us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:56:12.872859 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:56:12.874065 (+ 1206us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:56:12.874067 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:56:12.875478 (+ 1411us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:56:12.875490 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:56:12.876442 (+ 952us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:56:12.876446 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:56:12.876639 (+ 193us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:56:12.876714 (+ 75us) client_negotiation.cc:769] Sending connection context
0811 02:56:12.876787 (+ 73us) client_negotiation.cc:240] Negotiation successful
0811 02:56:12.876896 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":203,"mutex_wait_us":103,"spinlock_wait_cycles":8320,"thread_start_us":130,"threads_started":1}
I20260811 02:56:12.877972 3849 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "013b5b3a9dfb4b4285cbcfd16ceb2751" candidate_uuid: "ebd8dfc93d8a4aa998fff3349dd5c8d5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b0636fd10ece4f0ab0ac6b42437d4104" is_pre_election: true
I20260811 02:56:12.877978 3690 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "013b5b3a9dfb4b4285cbcfd16ceb2751" candidate_uuid: "ebd8dfc93d8a4aa998fff3349dd5c8d5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5b9aa0c640264b13b9de4aaec74004e4" is_pre_election: true
I20260811 02:56:12.878301 3849 raft_consensus.cc:2454] T 013b5b3a9dfb4b4285cbcfd16ceb2751 P b0636fd10ece4f0ab0ac6b42437d4104 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ebd8dfc93d8a4aa998fff3349dd5c8d5 in term 0.
I20260811 02:56:12.878309 3690 raft_consensus.cc:2454] T 013b5b3a9dfb4b4285cbcfd16ceb2751 P 5b9aa0c640264b13b9de4aaec74004e4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ebd8dfc93d8a4aa998fff3349dd5c8d5 in term 0.
I20260811 02:56:12.878877 3468 leader_election.cc:302] T 013b5b3a9dfb4b4285cbcfd16ceb2751 P ebd8dfc93d8a4aa998fff3349dd5c8d5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b0636fd10ece4f0ab0ac6b42437d4104, ebd8dfc93d8a4aa998fff3349dd5c8d5; no voters:
I20260811 02:56:12.879205 3928 raft_consensus.cc:2781] T 013b5b3a9dfb4b4285cbcfd16ceb2751 P ebd8dfc93d8a4aa998fff3349dd5c8d5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 02:56:12.879297 3928 raft_consensus.cc:482] T 013b5b3a9dfb4b4285cbcfd16ceb2751 P ebd8dfc93d8a4aa998fff3349dd5c8d5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 02:56:12.879329 3928 raft_consensus.cc:3037] T 013b5b3a9dfb4b4285cbcfd16ceb2751 P ebd8dfc93d8a4aa998fff3349dd5c8d5 [term 0 FOLLOWER]: Advancing to term 1
I20260811 02:56:12.880498 3928 raft_consensus.cc:504] T 013b5b3a9dfb4b4285cbcfd16ceb2751 P ebd8dfc93d8a4aa998fff3349dd5c8d5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b9aa0c640264b13b9de4aaec74004e4" member_type: VOTER last_known_addr { host: "127.12.138.194" port: 42409 } } peers { permanent_uuid: "ebd8dfc93d8a4aa998fff3349dd5c8d5" member_type: VOTER last_known_addr { host: "127.12.138.193" port: 36885 } } peers { permanent_uuid: "b0636fd10ece4f0ab0ac6b42437d4104" member_type: VOTER last_known_addr { host: "127.12.138.195" port: 35849 } }
I20260811 02:56:12.880954 3928 leader_election.cc:288] T 013b5b3a9dfb4b4285cbcfd16ceb2751 P ebd8dfc93d8a4aa998fff3349dd5c8d5 [CANDIDATE]: Term 1 election: Requested vote from peers 5b9aa0c640264b13b9de4aaec74004e4 (127.12.138.194:42409), b0636fd10ece4f0ab0ac6b42437d4104 (127.12.138.195:35849)
I20260811 02:56:12.881383 3690 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "013b5b3a9dfb4b4285cbcfd16ceb2751" candidate_uuid: "ebd8dfc93d8a4aa998fff3349dd5c8d5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5b9aa0c640264b13b9de4aaec74004e4"
I20260811 02:56:12.881472 3849 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "013b5b3a9dfb4b4285cbcfd16ceb2751" candidate_uuid: "ebd8dfc93d8a4aa998fff3349dd5c8d5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b0636fd10ece4f0ab0ac6b42437d4104"
I20260811 02:56:12.881529 3690 raft_consensus.cc:3037] T 013b5b3a9dfb4b4285cbcfd16ceb2751 P 5b9aa0c640264b13b9de4aaec74004e4 [term 0 FOLLOWER]: Advancing to term 1
I20260811 02:56:12.881711 3849 raft_consensus.cc:3037] T 013b5b3a9dfb4b4285cbcfd16ceb2751 P b0636fd10ece4f0ab0ac6b42437d4104 [term 0 FOLLOWER]: Advancing to term 1
I20260811 02:56:12.882762 3849 raft_consensus.cc:2454] T 013b5b3a9dfb4b4285cbcfd16ceb2751 P b0636fd10ece4f0ab0ac6b42437d4104 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ebd8dfc93d8a4aa998fff3349dd5c8d5 in term 1.
I20260811 02:56:12.882810 3690 raft_consensus.cc:2454] T 013b5b3a9dfb4b4285cbcfd16ceb2751 P 5b9aa0c640264b13b9de4aaec74004e4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ebd8dfc93d8a4aa998fff3349dd5c8d5 in term 1.
I20260811 02:56:12.883090 3468 leader_election.cc:302] T 013b5b3a9dfb4b4285cbcfd16ceb2751 P ebd8dfc93d8a4aa998fff3349dd5c8d5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b0636fd10ece4f0ab0ac6b42437d4104, ebd8dfc93d8a4aa998fff3349dd5c8d5; no voters:
I20260811 02:56:12.883275 3928 raft_consensus.cc:2781] T 013b5b3a9dfb4b4285cbcfd16ceb2751 P ebd8dfc93d8a4aa998fff3349dd5c8d5 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 02:56:12.883500 3928 raft_consensus.cc:686] T 013b5b3a9dfb4b4285cbcfd16ceb2751 P ebd8dfc93d8a4aa998fff3349dd5c8d5 [term 1 LEADER]: Becoming Leader. State: Replica: ebd8dfc93d8a4aa998fff3349dd5c8d5, State: Running, Role: LEADER
I20260811 02:56:12.883823 3928 consensus_queue.cc:237] T 013b5b3a9dfb4b4285cbcfd16ceb2751 P ebd8dfc93d8a4aa998fff3349dd5c8d5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b9aa0c640264b13b9de4aaec74004e4" member_type: VOTER last_known_addr { host: "127.12.138.194" port: 42409 } } peers { permanent_uuid: "ebd8dfc93d8a4aa998fff3349dd5c8d5" member_type: VOTER last_known_addr { host: "127.12.138.193" port: 36885 } } peers { permanent_uuid: "b0636fd10ece4f0ab0ac6b42437d4104" member_type: VOTER last_known_addr { host: "127.12.138.195" port: 35849 } }
I20260811 02:56:12.887038 3381 catalog_manager.cc:5495] T 013b5b3a9dfb4b4285cbcfd16ceb2751 P ebd8dfc93d8a4aa998fff3349dd5c8d5 reported cstate change: term changed from 0 to 1, leader changed from <none> to ebd8dfc93d8a4aa998fff3349dd5c8d5 (127.12.138.193). New cstate: current_term: 1 leader_uuid: "ebd8dfc93d8a4aa998fff3349dd5c8d5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b9aa0c640264b13b9de4aaec74004e4" member_type: VOTER last_known_addr { host: "127.12.138.194" port: 42409 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ebd8dfc93d8a4aa998fff3349dd5c8d5" member_type: VOTER last_known_addr { host: "127.12.138.193" port: 36885 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b0636fd10ece4f0ab0ac6b42437d4104" member_type: VOTER last_known_addr { host: "127.12.138.195" port: 35849 } health_report { overall_health: UNKNOWN } } }
I20260811 02:56:12.940758 3917 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:56:12.937471 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54114 (local address 127.12.138.193:36885)
0811 02:56:12.937600 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0811 02:56:12.937604 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 02:56:12.937676 (+ 72us) server_negotiation.cc:407] Connection header received
0811 02:56:12.937841 (+ 165us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:56:12.937843 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:56:12.937904 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:56:12.937994 (+ 90us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 02:56:12.938158 (+ 164us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:56:12.938653 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:56:12.939490 (+ 837us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:56:12.939693 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:56:12.939770 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:56:12.940022 (+ 252us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 02:56:12.940450 (+ 428us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 02:56:12.940542 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0811 02:56:12.940595 (+ 53us) server_negotiation.cc:299] Negotiation successful
0811 02:56:12.940642 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
W20260811 02:56:12.945916 3904 tablet.cc:2369] T 013b5b3a9dfb4b4285cbcfd16ceb2751 P b0636fd10ece4f0ab0ac6b42437d4104: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 02:56:12.946928 3690 raft_consensus.cc:1261] T 013b5b3a9dfb4b4285cbcfd16ceb2751 P 5b9aa0c640264b13b9de4aaec74004e4 [term 1 FOLLOWER]: Refusing update from remote peer ebd8dfc93d8a4aa998fff3349dd5c8d5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 02:56:12.946928 3849 raft_consensus.cc:1261] T 013b5b3a9dfb4b4285cbcfd16ceb2751 P b0636fd10ece4f0ab0ac6b42437d4104 [term 1 FOLLOWER]: Refusing update from remote peer ebd8dfc93d8a4aa998fff3349dd5c8d5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 02:56:12.947708 3933 consensus_queue.cc:1035] T 013b5b3a9dfb4b4285cbcfd16ceb2751 P ebd8dfc93d8a4aa998fff3349dd5c8d5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b0636fd10ece4f0ab0ac6b42437d4104" member_type: VOTER last_known_addr { host: "127.12.138.195" port: 35849 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 02:56:12.947944 3928 consensus_queue.cc:1035] T 013b5b3a9dfb4b4285cbcfd16ceb2751 P ebd8dfc93d8a4aa998fff3349dd5c8d5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5b9aa0c640264b13b9de4aaec74004e4" member_type: VOTER last_known_addr { host: "127.12.138.194" port: 42409 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 02:56:12.958683 3938 mvcc.cc:205] Tried to move back new op lower bound from 7317163921182420992 to 7317163920934658048. Current Snapshot: MvccSnapshot[applied={T|T < 7317163921182420992}]
I20260811 02:56:12.958915 3940 mvcc.cc:205] Tried to move back new op lower bound from 7317163921182420992 to 7317163920934658048. Current Snapshot: MvccSnapshot[applied={T|T < 7317163921182420992}]
I20260811 02:56:12.963163 3381 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:48634:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 02:56:12.963347 3381 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:48634:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 02:56:12.966048 3381 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 8b5b4e318d3a43849e86e88ba9d867e8: Sending DeleteTablet for 3 replicas of tablet 013b5b3a9dfb4b4285cbcfd16ceb2751
I20260811 02:56:12.966877 3665 tablet_service.cc:1514] Processing DeleteTablet for tablet 013b5b3a9dfb4b4285cbcfd16ceb2751 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 02:56:12 UTC) from {username='slave'} at 127.0.0.1:37956
I20260811 02:56:12.966984 3824 tablet_service.cc:1514] Processing DeleteTablet for tablet 013b5b3a9dfb4b4285cbcfd16ceb2751 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 02:56:12 UTC) from {username='slave'} at 127.0.0.1:45752
I20260811 02:56:12.967520 3948 tablet_replica.cc:330] T 013b5b3a9dfb4b4285cbcfd16ceb2751 P b0636fd10ece4f0ab0ac6b42437d4104: stopping tablet replica
I20260811 02:56:12.967559 12843 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDawEwS/build/debug/bin/kudu with pid 3451
I20260811 02:56:12.967934 3948 raft_consensus.cc:2229] T 013b5b3a9dfb4b4285cbcfd16ceb2751 P b0636fd10ece4f0ab0ac6b42437d4104 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 02:56:12.968406 3948 raft_consensus.cc:2258] T 013b5b3a9dfb4b4285cbcfd16ceb2751 P b0636fd10ece4f0ab0ac6b42437d4104 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 02:56:12.970892 3947 tablet_replica.cc:330] T 013b5b3a9dfb4b4285cbcfd16ceb2751 P 5b9aa0c640264b13b9de4aaec74004e4: stopping tablet replica
I20260811 02:56:12.971287 3947 raft_consensus.cc:2229] T 013b5b3a9dfb4b4285cbcfd16ceb2751 P 5b9aa0c640264b13b9de4aaec74004e4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 02:56:12.971729 3947 raft_consensus.cc:2258] T 013b5b3a9dfb4b4285cbcfd16ceb2751 P 5b9aa0c640264b13b9de4aaec74004e4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 02:56:12.972276 3948 ts_tablet_manager.cc:1905] T 013b5b3a9dfb4b4285cbcfd16ceb2751 P b0636fd10ece4f0ab0ac6b42437d4104: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 02:56:12.973376 3947 ts_tablet_manager.cc:1905] T 013b5b3a9dfb4b4285cbcfd16ceb2751 P 5b9aa0c640264b13b9de4aaec74004e4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 02:56:12.975268 3948 ts_tablet_manager.cc:1918] T 013b5b3a9dfb4b4285cbcfd16ceb2751 P b0636fd10ece4f0ab0ac6b42437d4104: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 02:56:12.975387 3948 log.cc:1196] T 013b5b3a9dfb4b4285cbcfd16ceb2751 P b0636fd10ece4f0ab0ac6b42437d4104: Deleting WAL directory at /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786416707705494-12843-0/minicluster-data/ts-2/wal/wals/013b5b3a9dfb4b4285cbcfd16ceb2751
I20260811 02:56:12.975698 3948 ts_tablet_manager.cc:1939] T 013b5b3a9dfb4b4285cbcfd16ceb2751 P b0636fd10ece4f0ab0ac6b42437d4104: Deleting consensus metadata
I20260811 02:56:12.976274 3947 ts_tablet_manager.cc:1918] T 013b5b3a9dfb4b4285cbcfd16ceb2751 P 5b9aa0c640264b13b9de4aaec74004e4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 02:56:12.976372 3947 log.cc:1196] T 013b5b3a9dfb4b4285cbcfd16ceb2751 P 5b9aa0c640264b13b9de4aaec74004e4: Deleting WAL directory at /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786416707705494-12843-0/minicluster-data/ts-1/wal/wals/013b5b3a9dfb4b4285cbcfd16ceb2751
I20260811 02:56:12.976682 3947 ts_tablet_manager.cc:1939] T 013b5b3a9dfb4b4285cbcfd16ceb2751 P 5b9aa0c640264b13b9de4aaec74004e4: Deleting consensus metadata
I20260811 02:56:12.977188 3366 catalog_manager.cc:4841] TS b0636fd10ece4f0ab0ac6b42437d4104 (127.12.138.195:35849): tablet 013b5b3a9dfb4b4285cbcfd16ceb2751 (table test-table [id=86da6fc5167b4fa0ac4e996150715549]) successfully deleted
I20260811 02:56:12.977895 3366 catalog_manager.cc:4841] TS 5b9aa0c640264b13b9de4aaec74004e4 (127.12.138.194:42409): tablet 013b5b3a9dfb4b4285cbcfd16ceb2751 (table test-table [id=86da6fc5167b4fa0ac4e996150715549]) successfully deleted
W20260811 02:56:12.978857 3366 connection.cc:535] client connection to 127.12.138.193:36885 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260811 02:56:12.979019 3366 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260811 02:56:12.979789 12843 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDawEwS/build/debug/bin/kudu with pid 3595
I20260811 02:56:12.980036 3918 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:56:12.979730 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.138.193:36885 (local address 127.0.0.1:54128)
0811 02:56:12.979854 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0811 02:56:12.979934 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.138.193:36885: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
W20260811 02:56:12.984711 3366 catalog_manager.cc:4584] TS ebd8dfc93d8a4aa998fff3349dd5c8d5 (127.12.138.193:36885): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 013b5b3a9dfb4b4285cbcfd16ceb2751: Network error: Client connection negotiation failed: client connection to 127.12.138.193:36885: connect: Connection refused (error 111)
I20260811 02:56:12.989874 12843 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDawEwS/build/debug/bin/kudu with pid 3749
I20260811 02:56:12.999472 12843 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDawEwS/build/debug/bin/kudu with pid 3350
2026-08-11T02:56:13Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/5 (1213 ms)
[----------] 6 tests from EncryptionPolicyTest (11396 ms total)
[----------] 18 tests from AuthTokenIssuingTest
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/0
Loading random data
Initializing database '/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786416707705494-12843-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 02:56:13 dist-test-slave-9gcw krb5kdc[3952](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 02:56:13 dist-test-slave-9gcw krb5kdc[3952](info): set up 2 sockets
Aug 11 02:56:13 dist-test-slave-9gcw krb5kdc[3952](info): commencing operation
krb5kdc: starting...
W20260811 02:56:14.161231 12843 mini_kdc.cc:121] Time spent starting KDC: real 1.137s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786416707705494-12843-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786416707705494-12843-0/krb5kdc/test-user.keytab.
Aug 11 02:56:14 dist-test-slave-9gcw krb5kdc[3952](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416974, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T02:56:14Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T02:56:14Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786416707705494-12843-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786416707705494-12843-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260811 02:56:14.300411 12843 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDawEwS/build/debug/bin/kudu
/tmp/dist-test-taskDawEwS/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786416707705494-12843-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786416707705494-12843-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786416707705494-12843-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786416707705494-12843-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42987
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786416707705494-12843-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:39547
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:42987
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786416707705494-12843-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786416707705494-12843-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786416707705494-12843-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 02:56:14.415465 3966 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 02:56:14.415796 3966 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 02:56:14.415884 3966 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 02:56:14.419868 3966 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 02:56:14.419962 3966 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 02:56:14.420002 3966 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 02:56:14.420039 3966 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 02:56:14.420071 3966 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 02:56:14.425345 3966 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:39547
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786416707705494-12843-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786416707705494-12843-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:42987
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786416707705494-12843-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42987
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786416707705494-12843-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3966
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786416707705494-12843-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 02:09:49 UTC on ccycloud.quasar-fsnqqc.root.comops.site
build id 4579570
I20260811 02:56:14.426405 3966 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 02:56:14.427431 3966 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 02:56:14.436223 3972 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:56:14.436189 3973 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:56:14.436313 3966 server_base.cc:1034] running on GCE node
W20260811 02:56:14.436189 3975 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:56:14.437217 3966 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 02:56:14.438408 3966 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 02:56:14.439584 3966 hybrid_clock.cc:648] HybridClock initialized: now 1786416974439571 us; error 27 us; skew 500 ppm
Aug 11 02:56:14 dist-test-slave-9gcw krb5kdc[3952](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416974, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 02:56:14.442802 3966 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260811 02:56:14.443742 3966 webserver.cc:460] Webserver started at http://127.0.0.1:46237/ using document root <none> and password file <none>
I20260811 02:56:14.444350 3966 fs_manager.cc:359] Metadata directory not provided
I20260811 02:56:14.444442 3966 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 02:56:14.444689 3966 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 02:56:14.446863 3966 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786416707705494-12843-0/minicluster-data/master-0/data/instance:
uuid: "d0e3cf487b0a4f2a87a56a050c5775a5"
format_stamp: "Formatted at 2026-08-11 02:56:14 on dist-test-slave-9gcw"
server_key: "6e1e7143c8dc322e6b0a8e6629e81db5"
server_key_iv: "48073d5b1d311a914753dea57ea0e758"
server_key_version: "encryptionkey@0"
I20260811 02:56:14.447511 3966 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786416707705494-12843-0/minicluster-data/master-0/wal/instance:
uuid: "d0e3cf487b0a4f2a87a56a050c5775a5"
format_stamp: "Formatted at 2026-08-11 02:56:14 on dist-test-slave-9gcw"
server_key: "6e1e7143c8dc322e6b0a8e6629e81db5"
server_key_iv: "48073d5b1d311a914753dea57ea0e758"
server_key_version: "encryptionkey@0"
I20260811 02:56:14.451488 3966 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260811 02:56:14.454607 3983 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 02:56:14.456032 3966 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260811 02:56:14.456215 3966 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786416707705494-12843-0/minicluster-data/master-0/data,/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786416707705494-12843-0/minicluster-data/master-0/wal
uuid: "d0e3cf487b0a4f2a87a56a050c5775a5"
format_stamp: "Formatted at 2026-08-11 02:56:14 on dist-test-slave-9gcw"
server_key: "6e1e7143c8dc322e6b0a8e6629e81db5"
server_key_iv: "48073d5b1d311a914753dea57ea0e758"
server_key_version: "encryptionkey@0"
I20260811 02:56:14.456338 3966 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786416707705494-12843-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786416707705494-12843-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786416707705494-12843-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 02:56:14.478982 3966 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 02:56:14.482391 3966 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 02:56:14.482607 3966 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 02:56:14.489861 3966 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:42987
I20260811 02:56:14.489902 4040 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:42987 every 8 connection(s)
I20260811 02:56:14.490844 3966 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786416707705494-12843-0/minicluster-data/master-0/data/info.pb
I20260811 02:56:14.493759 4041 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 02:56:14.498443 12843 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDawEwS/build/debug/bin/kudu as pid 3966
I20260811 02:56:14.498582 12843 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786416707705494-12843-0/minicluster-data/master-0/wal/instance
I20260811 02:56:14.498919 12843 external_mini_cluster.cc:1428] Setting key 44345b69e2f618044120a44c03c2379f
I20260811 02:56:14.499841 4041 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d0e3cf487b0a4f2a87a56a050c5775a5: Bootstrap starting.
I20260811 02:56:14.502346 4041 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d0e3cf487b0a4f2a87a56a050c5775a5: Neither blocks nor log segments found. Creating new log.
I20260811 02:56:14.503087 4041 log.cc:824] T 00000000000000000000000000000000 P d0e3cf487b0a4f2a87a56a050c5775a5: Log is configured to *not* fsync() on all Append() calls
I20260811 02:56:14.505635 4041 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d0e3cf487b0a4f2a87a56a050c5775a5: No bootstrap required, opened a new log
Aug 11 02:56:14 dist-test-slave-9gcw krb5kdc[3952](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416974, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260811 02:56:14.509979 4041 raft_consensus.cc:348] T 00000000000000000000000000000000 P d0e3cf487b0a4f2a87a56a050c5775a5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d0e3cf487b0a4f2a87a56a050c5775a5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42987 } }
I20260811 02:56:14.510300 4041 raft_consensus.cc:374] T 00000000000000000000000000000000 P d0e3cf487b0a4f2a87a56a050c5775a5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 02:56:14.510380 4041 raft_consensus.cc:729] T 00000000000000000000000000000000 P d0e3cf487b0a4f2a87a56a050c5775a5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d0e3cf487b0a4f2a87a56a050c5775a5, State: Initialized, Role: FOLLOWER
I20260811 02:56:14.510993 4041 consensus_queue.cc:260] T 00000000000000000000000000000000 P d0e3cf487b0a4f2a87a56a050c5775a5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d0e3cf487b0a4f2a87a56a050c5775a5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42987 } }
I20260811 02:56:14.511198 4041 raft_consensus.cc:388] T 00000000000000000000000000000000 P d0e3cf487b0a4f2a87a56a050c5775a5 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 02:56:14.511292 4041 raft_consensus.cc:482] T 00000000000000000000000000000000 P d0e3cf487b0a4f2a87a56a050c5775a5 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 02:56:14.511410 4041 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d0e3cf487b0a4f2a87a56a050c5775a5 [term 0 FOLLOWER]: Advancing to term 1
I20260811 02:56:14.512401 4041 raft_consensus.cc:504] T 00000000000000000000000000000000 P d0e3cf487b0a4f2a87a56a050c5775a5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d0e3cf487b0a4f2a87a56a050c5775a5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42987 } }
I20260811 02:56:14.512786 4041 leader_election.cc:302] T 00000000000000000000000000000000 P d0e3cf487b0a4f2a87a56a050c5775a5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d0e3cf487b0a4f2a87a56a050c5775a5; no voters:
I20260811 02:56:14.513230 4041 leader_election.cc:288] T 00000000000000000000000000000000 P d0e3cf487b0a4f2a87a56a050c5775a5 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 02:56:14.513353 4047 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d0e3cf487b0a4f2a87a56a050c5775a5 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 02:56:14.513643 4047 raft_consensus.cc:686] T 00000000000000000000000000000000 P d0e3cf487b0a4f2a87a56a050c5775a5 [term 1 LEADER]: Becoming Leader. State: Replica: d0e3cf487b0a4f2a87a56a050c5775a5, State: Running, Role: LEADER
I20260811 02:56:14.514015 4047 consensus_queue.cc:237] T 00000000000000000000000000000000 P d0e3cf487b0a4f2a87a56a050c5775a5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d0e3cf487b0a4f2a87a56a050c5775a5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42987 } }
I20260811 02:56:14.514772 4041 sys_catalog.cc:564] T 00000000000000000000000000000000 P d0e3cf487b0a4f2a87a56a050c5775a5 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 02:56:14.515897 4049 sys_catalog.cc:455] T 00000000000000000000000000000000 P d0e3cf487b0a4f2a87a56a050c5775a5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d0e3cf487b0a4f2a87a56a050c5775a5. Latest consensus state: current_term: 1 leader_uuid: "d0e3cf487b0a4f2a87a56a050c5775a5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d0e3cf487b0a4f2a87a56a050c5775a5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42987 } } }
I20260811 02:56:14.516050 4049 sys_catalog.cc:458] T 00000000000000000000000000000000 P d0e3cf487b0a4f2a87a56a050c5775a5 [sys.catalog]: This master's current role is: LEADER
I20260811 02:56:14.516489 4048 sys_catalog.cc:455] T 00000000000000000000000000000000 P d0e3cf487b0a4f2a87a56a050c5775a5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d0e3cf487b0a4f2a87a56a050c5775a5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d0e3cf487b0a4f2a87a56a050c5775a5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42987 } } }
I20260811 02:56:14.516621 4048 sys_catalog.cc:458] T 00000000000000000000000000000000 P d0e3cf487b0a4f2a87a56a050c5775a5 [sys.catalog]: This master's current role is: LEADER
I20260811 02:56:14.516489 4059 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 02:56:14.516996 4044 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:56:14.500514 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54975 (local address 127.0.0.1:42987)
0811 02:56:14.500977 (+ 463us) server_negotiation.cc:206] Beginning negotiation
0811 02:56:14.500994 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0811 02:56:14.501033 (+ 39us) server_negotiation.cc:407] Connection header received
0811 02:56:14.501862 (+ 829us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:56:14.501880 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:56:14.502266 (+ 386us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:56:14.502661 (+ 395us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:56:14.503664 (+ 1003us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:56:14.505405 (+ 1741us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:56:14.506402 (+ 997us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:56:14.507163 (+ 761us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:56:14.507347 (+ 184us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:56:14.510030 (+ 2683us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:56:14.510047 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:56:14.510065 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:56:14.510100 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:56:14.512370 (+ 2270us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:56:14.512959 (+ 589us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:56:14.512966 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:56:14.512977 (+ 11us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:56:14.513113 (+ 136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:56:14.513620 (+ 507us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:56:14.513623 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:56:14.513625 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:56:14.514225 (+ 600us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:56:14.514448 (+ 223us) server_negotiation.cc:1035] Waiting for connection context
0811 02:56:14.515505 (+ 1057us) server_negotiation.cc:299] Negotiation successful
0811 02:56:14.515789 (+ 284us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":309,"thread_start_us":142,"threads_started":1}
I20260811 02:56:14.520399 4059 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 02:56:14.520668 12843 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 02:56:14.526660 4059 catalog_manager.cc:1347] Generated new cluster ID: c8f8406b6ff64fea8a330e1caa298cd7
I20260811 02:56:14.526727 4059 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 02:56:14.533620 4044 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:56:14.523198 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59555 (local address 127.0.0.1:42987)
0811 02:56:14.523321 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0811 02:56:14.523326 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 02:56:14.523634 (+ 308us) server_negotiation.cc:407] Connection header received
0811 02:56:14.523780 (+ 146us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:56:14.523783 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:56:14.523835 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:56:14.523930 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:56:14.524871 (+ 941us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:56:14.525815 (+ 944us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:56:14.527201 (+ 1386us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:56:14.527569 (+ 368us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:56:14.527690 (+ 121us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:56:14.529539 (+ 1849us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:56:14.529542 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:56:14.529545 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:56:14.529571 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:56:14.531384 (+ 1813us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:56:14.531977 (+ 593us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:56:14.531980 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:56:14.531983 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:56:14.532069 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:56:14.532460 (+ 391us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:56:14.532463 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:56:14.532465 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:56:14.532672 (+ 207us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:56:14.532808 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0811 02:56:14.533186 (+ 378us) server_negotiation.cc:299] Negotiation successful
0811 02:56:14.533285 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260811 02:56:14.546835 4059 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 02:56:14.547842 4059 catalog_manager.cc:1504] Loading token signing keys...
I20260811 02:56:14.554586 4059 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d0e3cf487b0a4f2a87a56a050c5775a5: Generated new TSK 0
I20260811 02:56:14.555521 4059 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 02:56:14.566468 12843 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDawEwS/build/debug/bin/kudu with pid 3966
W20260811 02:56:14.575970 3950 connection.cc:535] client connection to 127.0.0.1:42987 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:42987): Connection reset by peer (error 104)
2026-08-11T02:56:14Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/0 (1566 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/1
Loading random data
Initializing database '/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786416707705494-12843-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 02:56:14 dist-test-slave-9gcw krb5kdc[4079](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 02:56:14 dist-test-slave-9gcw krb5kdc[4079](info): set up 2 sockets
Aug 11 02:56:14 dist-test-slave-9gcw krb5kdc[4079](info): commencing operation
krb5kdc: starting...
W20260811 02:56:15.811338 12843 mini_kdc.cc:121] Time spent starting KDC: real 1.220s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786416707705494-12843-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786416707705494-12843-0/krb5kdc/test-user.keytab.
Aug 11 02:56:15 dist-test-slave-9gcw krb5kdc[4079](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416975, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T02:56:15Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T02:56:15Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786416707705494-12843-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786416707705494-12843-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260811 02:56:15.952194 12843 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDawEwS/build/debug/bin/kudu
/tmp/dist-test-taskDawEwS/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786416707705494-12843-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786416707705494-12843-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786416707705494-12843-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786416707705494-12843-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44723
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786416707705494-12843-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:36293
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:44723
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786416707705494-12843-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786416707705494-12843-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786416707705494-12843-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 02:56:16.073457 4095 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 02:56:16.073804 4095 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 02:56:16.073901 4095 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 02:56:16.078063 4095 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 02:56:16.078148 4095 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 02:56:16.078186 4095 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 02:56:16.078243 4095 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 02:56:16.078281 4095 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 02:56:16.083679 4095 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:36293
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786416707705494-12843-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786416707705494-12843-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:44723
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786416707705494-12843-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44723
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786416707705494-12843-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4095
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786416707705494-12843-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 02:09:49 UTC on ccycloud.quasar-fsnqqc.root.comops.site
build id 4579570
I20260811 02:56:16.084779 4095 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 02:56:16.085887 4095 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260811 02:56:16.095072 4095 server_base.cc:1034] running on GCE node
W20260811 02:56:16.095022 4101 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:56:16.095039 4103 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:56:16.095022 4100 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:56:16.095885 4095 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 02:56:16.097007 4095 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 02:56:16.098198 4095 hybrid_clock.cc:648] HybridClock initialized: now 1786416976098175 us; error 37 us; skew 500 ppm
Aug 11 02:56:16 dist-test-slave-9gcw krb5kdc[4079](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416976, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 02:56:16.105234 4095 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260811 02:56:16.106295 4095 webserver.cc:460] Webserver started at http://127.0.0.1:41423/ using document root <none> and password file <none>
I20260811 02:56:16.106896 4095 fs_manager.cc:359] Metadata directory not provided
I20260811 02:56:16.106992 4095 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 02:56:16.107234 4095 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 02:56:16.109501 4095 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786416707705494-12843-0/minicluster-data/master-0/data/instance:
uuid: "d1d39657870e46ba8b01bede1a112ed6"
format_stamp: "Formatted at 2026-08-11 02:56:16 on dist-test-slave-9gcw"
server_key: "26f66f55f903c8e10de7fedda195a3df"
server_key_iv: "a216b0b04bc0c54c81e1d34e47ce7161"
server_key_version: "encryptionkey@0"
I20260811 02:56:16.110105 4095 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786416707705494-12843-0/minicluster-data/master-0/wal/instance:
uuid: "d1d39657870e46ba8b01bede1a112ed6"
format_stamp: "Formatted at 2026-08-11 02:56:16 on dist-test-slave-9gcw"
server_key: "26f66f55f903c8e10de7fedda195a3df"
server_key_iv: "a216b0b04bc0c54c81e1d34e47ce7161"
server_key_version: "encryptionkey@0"
I20260811 02:56:16.114071 4095 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260811 02:56:16.116603 4110 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 02:56:16.117985 4095 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260811 02:56:16.118163 4095 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786416707705494-12843-0/minicluster-data/master-0/data,/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786416707705494-12843-0/minicluster-data/master-0/wal
uuid: "d1d39657870e46ba8b01bede1a112ed6"
format_stamp: "Formatted at 2026-08-11 02:56:16 on dist-test-slave-9gcw"
server_key: "26f66f55f903c8e10de7fedda195a3df"
server_key_iv: "a216b0b04bc0c54c81e1d34e47ce7161"
server_key_version: "encryptionkey@0"
I20260811 02:56:16.118285 4095 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786416707705494-12843-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786416707705494-12843-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786416707705494-12843-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 02:56:16.135537 4095 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 02:56:16.139312 4095 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 02:56:16.139587 4095 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 02:56:16.148731 4095 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:44723
I20260811 02:56:16.148829 4169 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:44723 every 8 connection(s)
I20260811 02:56:16.149994 4095 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786416707705494-12843-0/minicluster-data/master-0/data/info.pb
I20260811 02:56:16.153061 4170 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 02:56:16.159206 4170 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d1d39657870e46ba8b01bede1a112ed6: Bootstrap starting.
I20260811 02:56:16.160082 12843 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDawEwS/build/debug/bin/kudu as pid 4095
I20260811 02:56:16.160239 12843 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786416707705494-12843-0/minicluster-data/master-0/wal/instance
I20260811 02:56:16.160567 12843 external_mini_cluster.cc:1428] Setting key 0cdc457fd329e2cb27cdd4f78bbf89f5
I20260811 02:56:16.162438 4170 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d1d39657870e46ba8b01bede1a112ed6: Neither blocks nor log segments found. Creating new log.
I20260811 02:56:16.163327 4170 log.cc:824] T 00000000000000000000000000000000 P d1d39657870e46ba8b01bede1a112ed6: Log is configured to *not* fsync() on all Append() calls
I20260811 02:56:16.165668 4170 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d1d39657870e46ba8b01bede1a112ed6: No bootstrap required, opened a new log
I20260811 02:56:16.170382 4170 raft_consensus.cc:348] T 00000000000000000000000000000000 P d1d39657870e46ba8b01bede1a112ed6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d1d39657870e46ba8b01bede1a112ed6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44723 } }
I20260811 02:56:16.170687 4170 raft_consensus.cc:374] T 00000000000000000000000000000000 P d1d39657870e46ba8b01bede1a112ed6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 02:56:16.170780 4170 raft_consensus.cc:729] T 00000000000000000000000000000000 P d1d39657870e46ba8b01bede1a112ed6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d1d39657870e46ba8b01bede1a112ed6, State: Initialized, Role: FOLLOWER
I20260811 02:56:16.171181 4173 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:56:16.161979 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44123 (local address 127.0.0.1:44723)
0811 02:56:16.162476 (+ 497us) server_negotiation.cc:206] Beginning negotiation
0811 02:56:16.162492 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0811 02:56:16.162530 (+ 38us) server_negotiation.cc:407] Connection header received
0811 02:56:16.163286 (+ 756us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:56:16.163305 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:56:16.163763 (+ 458us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:56:16.164195 (+ 432us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:56:16.164592 (+ 397us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:56:16.165754 (+ 1162us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:56:16.167110 (+ 1356us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:56:16.167746 (+ 636us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:56:16.167892 (+ 146us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:56:16.169015 (+ 1123us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:56:16.169021 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:56:16.169040 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 02:56:16.169539 (+ 499us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:56:16.169675 (+ 136us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 02:56:16.169708 (+ 33us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:56:16.169835 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0811 02:56:16.170050 (+ 215us) server_negotiation.cc:299] Negotiation successful
0811 02:56:16.170210 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":346,"thread_start_us":164,"threads_started":1}
I20260811 02:56:16.171317 4170 consensus_queue.cc:260] T 00000000000000000000000000000000 P d1d39657870e46ba8b01bede1a112ed6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d1d39657870e46ba8b01bede1a112ed6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44723 } }
I20260811 02:56:16.171522 4170 raft_consensus.cc:388] T 00000000000000000000000000000000 P d1d39657870e46ba8b01bede1a112ed6 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 02:56:16.171610 4170 raft_consensus.cc:482] T 00000000000000000000000000000000 P d1d39657870e46ba8b01bede1a112ed6 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 02:56:16.171708 4170 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d1d39657870e46ba8b01bede1a112ed6 [term 0 FOLLOWER]: Advancing to term 1
I20260811 02:56:16.172938 4170 raft_consensus.cc:504] T 00000000000000000000000000000000 P d1d39657870e46ba8b01bede1a112ed6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d1d39657870e46ba8b01bede1a112ed6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44723 } }
I20260811 02:56:16.173350 4170 leader_election.cc:302] T 00000000000000000000000000000000 P d1d39657870e46ba8b01bede1a112ed6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d1d39657870e46ba8b01bede1a112ed6; no voters:
I20260811 02:56:16.173697 4170 leader_election.cc:288] T 00000000000000000000000000000000 P d1d39657870e46ba8b01bede1a112ed6 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 02:56:16.173826 4176 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d1d39657870e46ba8b01bede1a112ed6 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 02:56:16.174070 4176 raft_consensus.cc:686] T 00000000000000000000000000000000 P d1d39657870e46ba8b01bede1a112ed6 [term 1 LEADER]: Becoming Leader. State: Replica: d1d39657870e46ba8b01bede1a112ed6, State: Running, Role: LEADER
I20260811 02:56:16.174459 4176 consensus_queue.cc:237] T 00000000000000000000000000000000 P d1d39657870e46ba8b01bede1a112ed6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d1d39657870e46ba8b01bede1a112ed6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44723 } }
I20260811 02:56:16.174853 12843 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 02:56:16.175150 4170 sys_catalog.cc:564] T 00000000000000000000000000000000 P d1d39657870e46ba8b01bede1a112ed6 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 02:56:16.176151 4179 sys_catalog.cc:455] T 00000000000000000000000000000000 P d1d39657870e46ba8b01bede1a112ed6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d1d39657870e46ba8b01bede1a112ed6. Latest consensus state: current_term: 1 leader_uuid: "d1d39657870e46ba8b01bede1a112ed6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d1d39657870e46ba8b01bede1a112ed6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44723 } } }
I20260811 02:56:16.176275 4179 sys_catalog.cc:458] T 00000000000000000000000000000000 P d1d39657870e46ba8b01bede1a112ed6 [sys.catalog]: This master's current role is: LEADER
I20260811 02:56:16.176633 4177 sys_catalog.cc:455] T 00000000000000000000000000000000 P d1d39657870e46ba8b01bede1a112ed6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d1d39657870e46ba8b01bede1a112ed6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d1d39657870e46ba8b01bede1a112ed6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44723 } } }
I20260811 02:56:16.176730 4177 sys_catalog.cc:458] T 00000000000000000000000000000000 P d1d39657870e46ba8b01bede1a112ed6 [sys.catalog]: This master's current role is: LEADER
I20260811 02:56:16.177039 4190 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 02:56:16.180797 4190 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 02:56:16.185189 4173 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:56:16.177563 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47073 (local address 127.0.0.1:44723)
0811 02:56:16.177733 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0811 02:56:16.177738 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 02:56:16.178240 (+ 502us) server_negotiation.cc:407] Connection header received
0811 02:56:16.178416 (+ 176us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:56:16.178420 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:56:16.178472 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:56:16.178592 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:56:16.179067 (+ 475us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:56:16.180078 (+ 1011us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:56:16.181255 (+ 1177us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:56:16.181663 (+ 408us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:56:16.181791 (+ 128us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:56:16.184508 (+ 2717us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:56:16.184512 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:56:16.184515 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 02:56:16.184530 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:56:16.184564 (+ 34us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 02:56:16.184576 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:56:16.184689 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0811 02:56:16.184960 (+ 271us) server_negotiation.cc:299] Negotiation successful
0811 02:56:16.185004 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":95}
I20260811 02:56:16.186764 4190 catalog_manager.cc:1347] Generated new cluster ID: ea802fe14862495998d8158bff7987b0
I20260811 02:56:16.186838 4190 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 02:56:16.195211 4190 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 02:56:16.196233 4190 catalog_manager.cc:1504] Loading token signing keys...
I20260811 02:56:16.201203 4190 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d1d39657870e46ba8b01bede1a112ed6: Generated new TSK 0
I20260811 02:56:16.201872 4190 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 02:56:16.208238 12843 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDawEwS/build/debug/bin/kudu with pid 4095
W20260811 02:56:16.218374 4077 connection.cc:535] client connection to 127.0.0.1:44723 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:44723): Connection reset by peer (error 104)
2026-08-11T02:56:16Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/1 (1641 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/2
Loading random data
Initializing database '/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786416707705494-12843-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 02:56:16 dist-test-slave-9gcw krb5kdc[4206](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 02:56:16 dist-test-slave-9gcw krb5kdc[4206](info): set up 2 sockets
Aug 11 02:56:16 dist-test-slave-9gcw krb5kdc[4206](info): commencing operation
krb5kdc: starting...
W20260811 02:56:17.406424 12843 mini_kdc.cc:121] Time spent starting KDC: real 1.173s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786416707705494-12843-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786416707705494-12843-0/krb5kdc/test-user.keytab.
Aug 11 02:56:17 dist-test-slave-9gcw krb5kdc[4206](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416977, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T02:56:17Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T02:56:17Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786416707705494-12843-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786416707705494-12843-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260811 02:56:17.544968 12843 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDawEwS/build/debug/bin/kudu
/tmp/dist-test-taskDawEwS/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786416707705494-12843-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786416707705494-12843-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786416707705494-12843-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786416707705494-12843-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43187
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786416707705494-12843-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:35623
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:43187
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786416707705494-12843-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786416707705494-12843-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786416707705494-12843-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 02:56:17.674576 4221 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 02:56:17.674945 4221 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 02:56:17.675055 4221 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 02:56:17.680028 4221 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 02:56:17.680145 4221 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 02:56:17.680189 4221 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 02:56:17.680234 4221 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 02:56:17.680275 4221 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 02:56:17.686023 4221 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:35623
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786416707705494-12843-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786416707705494-12843-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:43187
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786416707705494-12843-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43187
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786416707705494-12843-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4221
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786416707705494-12843-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 02:09:49 UTC on ccycloud.quasar-fsnqqc.root.comops.site
build id 4579570
I20260811 02:56:17.687180 4221 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 02:56:17.688261 4221 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 02:56:17.697507 4231 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:56:17.697568 4227 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:56:17.697543 4228 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:56:17.697737 4221 server_base.cc:1034] running on GCE node
I20260811 02:56:17.698603 4221 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 02:56:17.699790 4221 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 02:56:17.700968 4221 hybrid_clock.cc:648] HybridClock initialized: now 1786416977700948 us; error 32 us; skew 500 ppm
Aug 11 02:56:17 dist-test-slave-9gcw krb5kdc[4206](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416977, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 02:56:17.704031 4221 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260811 02:56:17.705036 4221 webserver.cc:460] Webserver started at http://127.0.0.1:34393/ using document root <none> and password file <none>
I20260811 02:56:17.705657 4221 fs_manager.cc:359] Metadata directory not provided
I20260811 02:56:17.705761 4221 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 02:56:17.706007 4221 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 02:56:17.708106 4221 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786416707705494-12843-0/minicluster-data/master-0/data/instance:
uuid: "3e206eb4290f4a8583db3e5d4afed937"
format_stamp: "Formatted at 2026-08-11 02:56:17 on dist-test-slave-9gcw"
server_key: "69528a8eba0000592178859ec73dedb6"
server_key_iv: "8dccc1663a046286c012352f03bc34cf"
server_key_version: "encryptionkey@0"
I20260811 02:56:17.708729 4221 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786416707705494-12843-0/minicluster-data/master-0/wal/instance:
uuid: "3e206eb4290f4a8583db3e5d4afed937"
format_stamp: "Formatted at 2026-08-11 02:56:17 on dist-test-slave-9gcw"
server_key: "69528a8eba0000592178859ec73dedb6"
server_key_iv: "8dccc1663a046286c012352f03bc34cf"
server_key_version: "encryptionkey@0"
I20260811 02:56:17.712759 4221 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260811 02:56:17.715587 4238 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 02:56:17.716956 4221 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260811 02:56:17.717116 4221 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786416707705494-12843-0/minicluster-data/master-0/data,/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786416707705494-12843-0/minicluster-data/master-0/wal
uuid: "3e206eb4290f4a8583db3e5d4afed937"
format_stamp: "Formatted at 2026-08-11 02:56:17 on dist-test-slave-9gcw"
server_key: "69528a8eba0000592178859ec73dedb6"
server_key_iv: "8dccc1663a046286c012352f03bc34cf"
server_key_version: "encryptionkey@0"
I20260811 02:56:17.717299 4221 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786416707705494-12843-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786416707705494-12843-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786416707705494-12843-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 02:56:17.722447 4221 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 02:56:17.726157 4221 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 02:56:17.726379 4221 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 02:56:17.734548 4221 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:43187
I20260811 02:56:17.734606 4295 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:43187 every 8 connection(s)
I20260811 02:56:17.735599 4221 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786416707705494-12843-0/minicluster-data/master-0/data/info.pb
I20260811 02:56:17.738531 4296 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 02:56:17.742646 12843 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDawEwS/build/debug/bin/kudu as pid 4221
I20260811 02:56:17.742800 12843 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786416707705494-12843-0/minicluster-data/master-0/wal/instance
I20260811 02:56:17.743108 12843 external_mini_cluster.cc:1428] Setting key 4378a0a4902a2a730b52afb4ed17c79c
I20260811 02:56:17.744614 4296 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3e206eb4290f4a8583db3e5d4afed937: Bootstrap starting.
I20260811 02:56:17.746999 4296 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3e206eb4290f4a8583db3e5d4afed937: Neither blocks nor log segments found. Creating new log.
I20260811 02:56:17.747877 4296 log.cc:824] T 00000000000000000000000000000000 P 3e206eb4290f4a8583db3e5d4afed937: Log is configured to *not* fsync() on all Append() calls
I20260811 02:56:17.748683 4299 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:56:17.744577 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47049 (local address 127.0.0.1:43187)
0811 02:56:17.745090 (+ 513us) server_negotiation.cc:206] Beginning negotiation
0811 02:56:17.745113 (+ 23us) server_negotiation.cc:399] Waiting for connection header
0811 02:56:17.745172 (+ 59us) server_negotiation.cc:407] Connection header received
0811 02:56:17.745874 (+ 702us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:56:17.745893 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:56:17.746220 (+ 327us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:56:17.746579 (+ 359us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:56:17.747460 (+ 881us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:56:17.747464 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:56:17.747476 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 02:56:17.747487 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:56:17.747549 (+ 62us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 02:56:17.747569 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:56:17.747689 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0811 02:56:17.747809 (+ 120us) server_negotiation.cc:299] Negotiation successful
0811 02:56:17.747966 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":347,"thread_start_us":154,"threads_started":1}
I20260811 02:56:17.749970 4296 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3e206eb4290f4a8583db3e5d4afed937: No bootstrap required, opened a new log
I20260811 02:56:17.751783 12843 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 02:56:17.754166 4296 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3e206eb4290f4a8583db3e5d4afed937 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3e206eb4290f4a8583db3e5d4afed937" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43187 } }
I20260811 02:56:17.754423 4296 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3e206eb4290f4a8583db3e5d4afed937 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 02:56:17.754485 4296 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3e206eb4290f4a8583db3e5d4afed937 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3e206eb4290f4a8583db3e5d4afed937, State: Initialized, Role: FOLLOWER
I20260811 02:56:17.754952 4296 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3e206eb4290f4a8583db3e5d4afed937 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3e206eb4290f4a8583db3e5d4afed937" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43187 } }
I20260811 02:56:17.755146 4296 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3e206eb4290f4a8583db3e5d4afed937 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 02:56:17.755282 4296 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3e206eb4290f4a8583db3e5d4afed937 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 02:56:17.755446 4296 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3e206eb4290f4a8583db3e5d4afed937 [term 0 FOLLOWER]: Advancing to term 1
I20260811 02:56:17.755985 4299 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:56:17.754092 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47285 (local address 127.0.0.1:43187)
0811 02:56:17.754238 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0811 02:56:17.754241 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 02:56:17.754552 (+ 311us) server_negotiation.cc:407] Connection header received
0811 02:56:17.754628 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:56:17.754631 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:56:17.754666 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:56:17.754760 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:56:17.755558 (+ 798us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:56:17.755562 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:56:17.755565 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 02:56:17.755578 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:56:17.755609 (+ 31us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 02:56:17.755622 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:56:17.755716 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0811 02:56:17.755836 (+ 120us) server_negotiation.cc:299] Negotiation successful
0811 02:56:17.755874 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260811 02:56:17.756713 4296 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3e206eb4290f4a8583db3e5d4afed937 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3e206eb4290f4a8583db3e5d4afed937" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43187 } }
I20260811 02:56:17.757069 4296 leader_election.cc:302] T 00000000000000000000000000000000 P 3e206eb4290f4a8583db3e5d4afed937 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3e206eb4290f4a8583db3e5d4afed937; no voters:
I20260811 02:56:17.757418 4296 leader_election.cc:288] T 00000000000000000000000000000000 P 3e206eb4290f4a8583db3e5d4afed937 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 02:56:17.757545 4307 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3e206eb4290f4a8583db3e5d4afed937 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 02:56:17.758090 4307 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3e206eb4290f4a8583db3e5d4afed937 [term 1 LEADER]: Becoming Leader. State: Replica: 3e206eb4290f4a8583db3e5d4afed937, State: Running, Role: LEADER
I20260811 02:56:17.758509 4307 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3e206eb4290f4a8583db3e5d4afed937 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3e206eb4290f4a8583db3e5d4afed937" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43187 } }
I20260811 02:56:17.758806 4296 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3e206eb4290f4a8583db3e5d4afed937 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 02:56:17.760474 4308 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3e206eb4290f4a8583db3e5d4afed937 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3e206eb4290f4a8583db3e5d4afed937" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3e206eb4290f4a8583db3e5d4afed937" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43187 } } }
I20260811 02:56:17.760545 4309 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3e206eb4290f4a8583db3e5d4afed937 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3e206eb4290f4a8583db3e5d4afed937. Latest consensus state: current_term: 1 leader_uuid: "3e206eb4290f4a8583db3e5d4afed937" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3e206eb4290f4a8583db3e5d4afed937" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43187 } } }
I20260811 02:56:17.760679 4309 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3e206eb4290f4a8583db3e5d4afed937 [sys.catalog]: This master's current role is: LEADER
I20260811 02:56:17.760622 4308 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3e206eb4290f4a8583db3e5d4afed937 [sys.catalog]: This master's current role is: LEADER
W20260811 02:56:17.764159 4324 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 3e206eb4290f4a8583db3e5d4afed937: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 02:56:17.764262 4324 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 02:56:17.764343 4322 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 02:56:17.765040 4322 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 02:56:17.771365 4322 catalog_manager.cc:1347] Generated new cluster ID: b0d056e254954c9fb47fbacadd0d9d33
I20260811 02:56:17.771445 4322 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 02:56:17.784884 4322 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 02:56:17.785339 4322 catalog_manager.cc:1504] Loading token signing keys...
I20260811 02:56:17.790453 4322 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3e206eb4290f4a8583db3e5d4afed937: Generated new TSK 0
I20260811 02:56:17.791250 4322 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 02:56:17.799636 12843 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDawEwS/build/debug/bin/kudu with pid 4221
2026-08-11T02:56:17Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/2 (1592 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/3
Loading random data
Initializing database '/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786416707705494-12843-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 02:56:17 dist-test-slave-9gcw krb5kdc[4331](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 02:56:17 dist-test-slave-9gcw krb5kdc[4331](info): set up 2 sockets
Aug 11 02:56:17 dist-test-slave-9gcw krb5kdc[4331](info): commencing operation
krb5kdc: starting...
W20260811 02:56:18.936784 12843 mini_kdc.cc:121] Time spent starting KDC: real 1.111s user 0.003s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786416707705494-12843-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786416707705494-12843-0/krb5kdc/test-user.keytab.
Aug 11 02:56:19 dist-test-slave-9gcw krb5kdc[4331](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416979, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T02:56:19Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T02:56:19Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786416707705494-12843-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786416707705494-12843-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260811 02:56:19.081476 12843 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDawEwS/build/debug/bin/kudu
/tmp/dist-test-taskDawEwS/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786416707705494-12843-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786416707705494-12843-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786416707705494-12843-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786416707705494-12843-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43013
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786416707705494-12843-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:36983
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:43013
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786416707705494-12843-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786416707705494-12843-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786416707705494-12843-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 02:56:19.200176 4346 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 02:56:19.200472 4346 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 02:56:19.200518 4346 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 02:56:19.205009 4346 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 02:56:19.205076 4346 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 02:56:19.205098 4346 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 02:56:19.205117 4346 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 02:56:19.205137 4346 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 02:56:19.211068 4346 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:36983
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786416707705494-12843-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786416707705494-12843-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:43013
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786416707705494-12843-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43013
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786416707705494-12843-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4346
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786416707705494-12843-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 02:09:49 UTC on ccycloud.quasar-fsnqqc.root.comops.site
build id 4579570
I20260811 02:56:19.212183 4346 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 02:56:19.213269 4346 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260811 02:56:19.222846 4346 server_base.cc:1034] running on GCE node
W20260811 02:56:19.222741 4353 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:56:19.222867 4354 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:56:19.222764 4356 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:56:19.223651 4346 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 02:56:19.224890 4346 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 02:56:19.226095 4346 hybrid_clock.cc:648] HybridClock initialized: now 1786416979226086 us; error 65 us; skew 500 ppm
Aug 11 02:56:19 dist-test-slave-9gcw krb5kdc[4331](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416979, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 02:56:19.229014 4346 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260811 02:56:19.230016 4346 webserver.cc:460] Webserver started at http://127.0.0.1:45783/ using document root <none> and password file <none>
I20260811 02:56:19.230592 4346 fs_manager.cc:359] Metadata directory not provided
I20260811 02:56:19.230686 4346 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 02:56:19.230930 4346 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 02:56:19.233049 4346 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786416707705494-12843-0/minicluster-data/master-0/data/instance:
uuid: "f9c7dadf81df4f4c86363f95539ead0c"
format_stamp: "Formatted at 2026-08-11 02:56:19 on dist-test-slave-9gcw"
server_key: "72a29ccc466c8a8ad7dabeb396569f50"
server_key_iv: "63ce9f3cdab529cc062128bae590e55c"
server_key_version: "encryptionkey@0"
I20260811 02:56:19.233698 4346 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786416707705494-12843-0/minicluster-data/master-0/wal/instance:
uuid: "f9c7dadf81df4f4c86363f95539ead0c"
format_stamp: "Formatted at 2026-08-11 02:56:19 on dist-test-slave-9gcw"
server_key: "72a29ccc466c8a8ad7dabeb396569f50"
server_key_iv: "63ce9f3cdab529cc062128bae590e55c"
server_key_version: "encryptionkey@0"
I20260811 02:56:19.237509 4346 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260811 02:56:19.240026 4364 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 02:56:19.241364 4346 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260811 02:56:19.241503 4346 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786416707705494-12843-0/minicluster-data/master-0/data,/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786416707705494-12843-0/minicluster-data/master-0/wal
uuid: "f9c7dadf81df4f4c86363f95539ead0c"
format_stamp: "Formatted at 2026-08-11 02:56:19 on dist-test-slave-9gcw"
server_key: "72a29ccc466c8a8ad7dabeb396569f50"
server_key_iv: "63ce9f3cdab529cc062128bae590e55c"
server_key_version: "encryptionkey@0"
I20260811 02:56:19.241613 4346 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786416707705494-12843-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786416707705494-12843-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786416707705494-12843-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 02:56:19.283140 4346 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 02:56:19.286731 4346 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 02:56:19.286995 4346 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 02:56:19.295470 4346 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:43013
I20260811 02:56:19.295490 4422 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:43013 every 8 connection(s)
I20260811 02:56:19.296546 4346 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786416707705494-12843-0/minicluster-data/master-0/data/info.pb
I20260811 02:56:19.299754 4423 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 02:56:19.299779 12843 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDawEwS/build/debug/bin/kudu as pid 4346
I20260811 02:56:19.299926 12843 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786416707705494-12843-0/minicluster-data/master-0/wal/instance
I20260811 02:56:19.300323 12843 external_mini_cluster.cc:1428] Setting key 5888b6e66c46a0a0fdf09499bc7cb57a
I20260811 02:56:19.307263 4423 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f9c7dadf81df4f4c86363f95539ead0c: Bootstrap starting.
I20260811 02:56:19.310154 4423 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f9c7dadf81df4f4c86363f95539ead0c: Neither blocks nor log segments found. Creating new log.
Aug 11 02:56:19 dist-test-slave-9gcw krb5kdc[4331](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416979, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260811 02:56:19.310982 4423 log.cc:824] T 00000000000000000000000000000000 P f9c7dadf81df4f4c86363f95539ead0c: Log is configured to *not* fsync() on all Append() calls
I20260811 02:56:19.313442 4423 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f9c7dadf81df4f4c86363f95539ead0c: No bootstrap required, opened a new log
I20260811 02:56:19.316843 4426 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:56:19.301956 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36529 (local address 127.0.0.1:43013)
0811 02:56:19.302431 (+ 475us) server_negotiation.cc:206] Beginning negotiation
0811 02:56:19.302455 (+ 24us) server_negotiation.cc:399] Waiting for connection header
0811 02:56:19.302495 (+ 40us) server_negotiation.cc:407] Connection header received
0811 02:56:19.303309 (+ 814us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:56:19.303328 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:56:19.303693 (+ 365us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:56:19.304005 (+ 312us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:56:19.304996 (+ 991us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:56:19.306140 (+ 1144us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:56:19.307488 (+ 1348us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:56:19.308031 (+ 543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:56:19.308173 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:56:19.311536 (+ 3363us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:56:19.311541 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:56:19.311554 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:56:19.311579 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:56:19.313585 (+ 2006us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:56:19.314061 (+ 476us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:56:19.314068 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:56:19.314073 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:56:19.314221 (+ 148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:56:19.314490 (+ 269us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:56:19.314493 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:56:19.314495 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:56:19.314990 (+ 495us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:56:19.315203 (+ 213us) server_negotiation.cc:1035] Waiting for connection context
0811 02:56:19.315520 (+ 317us) server_negotiation.cc:299] Negotiation successful
0811 02:56:19.315870 (+ 350us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":321,"thread_start_us":185,"threads_started":1}
I20260811 02:56:19.318075 4423 raft_consensus.cc:348] T 00000000000000000000000000000000 P f9c7dadf81df4f4c86363f95539ead0c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f9c7dadf81df4f4c86363f95539ead0c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43013 } }
I20260811 02:56:19.318286 4423 raft_consensus.cc:374] T 00000000000000000000000000000000 P f9c7dadf81df4f4c86363f95539ead0c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 02:56:19.318343 4423 raft_consensus.cc:729] T 00000000000000000000000000000000 P f9c7dadf81df4f4c86363f95539ead0c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f9c7dadf81df4f4c86363f95539ead0c, State: Initialized, Role: FOLLOWER
I20260811 02:56:19.318806 4423 consensus_queue.cc:260] T 00000000000000000000000000000000 P f9c7dadf81df4f4c86363f95539ead0c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f9c7dadf81df4f4c86363f95539ead0c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43013 } }
I20260811 02:56:19.318962 4423 raft_consensus.cc:388] T 00000000000000000000000000000000 P f9c7dadf81df4f4c86363f95539ead0c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 02:56:19.319034 4423 raft_consensus.cc:482] T 00000000000000000000000000000000 P f9c7dadf81df4f4c86363f95539ead0c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 02:56:19.319134 4423 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f9c7dadf81df4f4c86363f95539ead0c [term 0 FOLLOWER]: Advancing to term 1
I20260811 02:56:19.320119 12843 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 02:56:19.320149 4423 raft_consensus.cc:504] T 00000000000000000000000000000000 P f9c7dadf81df4f4c86363f95539ead0c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f9c7dadf81df4f4c86363f95539ead0c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43013 } }
I20260811 02:56:19.320473 4423 leader_election.cc:302] T 00000000000000000000000000000000 P f9c7dadf81df4f4c86363f95539ead0c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f9c7dadf81df4f4c86363f95539ead0c; no voters:
I20260811 02:56:19.320818 4423 leader_election.cc:288] T 00000000000000000000000000000000 P f9c7dadf81df4f4c86363f95539ead0c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 02:56:19.320897 4428 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f9c7dadf81df4f4c86363f95539ead0c [term 1 FOLLOWER]: Leader election won for term 1
I20260811 02:56:19.321164 4428 raft_consensus.cc:686] T 00000000000000000000000000000000 P f9c7dadf81df4f4c86363f95539ead0c [term 1 LEADER]: Becoming Leader. State: Replica: f9c7dadf81df4f4c86363f95539ead0c, State: Running, Role: LEADER
I20260811 02:56:19.321508 4428 consensus_queue.cc:237] T 00000000000000000000000000000000 P f9c7dadf81df4f4c86363f95539ead0c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f9c7dadf81df4f4c86363f95539ead0c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43013 } }
I20260811 02:56:19.322127 4423 sys_catalog.cc:564] T 00000000000000000000000000000000 P f9c7dadf81df4f4c86363f95539ead0c [sys.catalog]: configured and running, proceeding with master startup.
I20260811 02:56:19.322747 4428 sys_catalog.cc:455] T 00000000000000000000000000000000 P f9c7dadf81df4f4c86363f95539ead0c [sys.catalog]: SysCatalogTable state changed. Reason: New leader f9c7dadf81df4f4c86363f95539ead0c. Latest consensus state: current_term: 1 leader_uuid: "f9c7dadf81df4f4c86363f95539ead0c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f9c7dadf81df4f4c86363f95539ead0c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43013 } } }
I20260811 02:56:19.322932 4428 sys_catalog.cc:458] T 00000000000000000000000000000000 P f9c7dadf81df4f4c86363f95539ead0c [sys.catalog]: This master's current role is: LEADER
I20260811 02:56:19.323312 4430 sys_catalog.cc:455] T 00000000000000000000000000000000 P f9c7dadf81df4f4c86363f95539ead0c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f9c7dadf81df4f4c86363f95539ead0c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f9c7dadf81df4f4c86363f95539ead0c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43013 } } }
I20260811 02:56:19.323398 4440 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 02:56:19.323428 4430 sys_catalog.cc:458] T 00000000000000000000000000000000 P f9c7dadf81df4f4c86363f95539ead0c [sys.catalog]: This master's current role is: LEADER
I20260811 02:56:19.326640 4440 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 02:56:19.333612 4440 catalog_manager.cc:1347] Generated new cluster ID: e3790b25f29b4f93a85e45a1f1a7b493
I20260811 02:56:19.333691 4440 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 02:56:19.334357 4426 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:56:19.324786 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59699 (local address 127.0.0.1:43013)
0811 02:56:19.324906 (+ 120us) server_negotiation.cc:206] Beginning negotiation
0811 02:56:19.324912 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 02:56:19.325073 (+ 161us) server_negotiation.cc:407] Connection header received
0811 02:56:19.325279 (+ 206us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:56:19.325286 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:56:19.325366 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:56:19.325470 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:56:19.326240 (+ 770us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:56:19.327144 (+ 904us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:56:19.328229 (+ 1085us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:56:19.328507 (+ 278us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:56:19.328629 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:56:19.330263 (+ 1634us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:56:19.330267 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:56:19.330270 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:56:19.330305 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:56:19.332267 (+ 1962us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:56:19.332783 (+ 516us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:56:19.332787 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:56:19.332789 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:56:19.332882 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:56:19.333249 (+ 367us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:56:19.333253 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:56:19.333255 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:56:19.333467 (+ 212us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:56:19.333628 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0811 02:56:19.334052 (+ 424us) server_negotiation.cc:299] Negotiation successful
0811 02:56:19.334188 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260811 02:56:19.348443 4440 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 02:56:19.350581 4440 catalog_manager.cc:1504] Loading token signing keys...
I20260811 02:56:19.364925 4440 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f9c7dadf81df4f4c86363f95539ead0c: Generated new TSK 0
I20260811 02:56:19.365729 4440 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 02:56:19.372289 12843 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDawEwS/build/debug/bin/kudu with pid 4346
2026-08-11T02:56:19Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/3 (1571 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/4
Loading random data
Initializing database '/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786416707705494-12843-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 02:56:19 dist-test-slave-9gcw krb5kdc[4457](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 02:56:19 dist-test-slave-9gcw krb5kdc[4457](info): set up 2 sockets
Aug 11 02:56:19 dist-test-slave-9gcw krb5kdc[4457](info): commencing operation
krb5kdc: starting...
W20260811 02:56:20.538451 12843 mini_kdc.cc:121] Time spent starting KDC: real 1.141s user 0.003s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786416707705494-12843-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786416707705494-12843-0/krb5kdc/test-user.keytab.
Aug 11 02:56:20 dist-test-slave-9gcw krb5kdc[4457](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416980, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T02:56:20Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T02:56:20Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786416707705494-12843-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786416707705494-12843-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260811 02:56:20.681249 12843 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDawEwS/build/debug/bin/kudu
/tmp/dist-test-taskDawEwS/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786416707705494-12843-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786416707705494-12843-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786416707705494-12843-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786416707705494-12843-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41517
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786416707705494-12843-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:36083
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:41517
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786416707705494-12843-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786416707705494-12843-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786416707705494-12843-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 02:56:20.798247 4474 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 02:56:20.798578 4474 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 02:56:20.798655 4474 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 02:56:20.802724 4474 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 02:56:20.802819 4474 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 02:56:20.802855 4474 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 02:56:20.802894 4474 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 02:56:20.802934 4474 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 02:56:20.808285 4474 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:36083
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786416707705494-12843-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786416707705494-12843-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:41517
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786416707705494-12843-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41517
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786416707705494-12843-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4474
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786416707705494-12843-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 02:09:49 UTC on ccycloud.quasar-fsnqqc.root.comops.site
build id 4579570
I20260811 02:56:20.809530 4474 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 02:56:20.810578 4474 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260811 02:56:20.819602 4474 server_base.cc:1034] running on GCE node
W20260811 02:56:20.819566 4481 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:56:20.819548 4483 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:56:20.819552 4485 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:56:20.820492 4474 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 02:56:20.821602 4474 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 02:56:20.822774 4474 hybrid_clock.cc:648] HybridClock initialized: now 1786416980822760 us; error 28 us; skew 500 ppm
Aug 11 02:56:20 dist-test-slave-9gcw krb5kdc[4457](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416980, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 02:56:20.825809 4474 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260811 02:56:20.826704 4474 webserver.cc:460] Webserver started at http://127.0.0.1:34891/ using document root <none> and password file <none>
I20260811 02:56:20.827310 4474 fs_manager.cc:359] Metadata directory not provided
I20260811 02:56:20.827410 4474 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 02:56:20.827656 4474 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 02:56:20.829790 4474 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786416707705494-12843-0/minicluster-data/master-0/data/instance:
uuid: "ee79fd986d7b447eb65d8c3fe8dc236a"
format_stamp: "Formatted at 2026-08-11 02:56:20 on dist-test-slave-9gcw"
server_key: "703ef7df7dc005dc6562f34fa3a41549"
server_key_iv: "b4bfcabe537b7ac4d665b594f20512d5"
server_key_version: "encryptionkey@0"
I20260811 02:56:20.830404 4474 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786416707705494-12843-0/minicluster-data/master-0/wal/instance:
uuid: "ee79fd986d7b447eb65d8c3fe8dc236a"
format_stamp: "Formatted at 2026-08-11 02:56:20 on dist-test-slave-9gcw"
server_key: "703ef7df7dc005dc6562f34fa3a41549"
server_key_iv: "b4bfcabe537b7ac4d665b594f20512d5"
server_key_version: "encryptionkey@0"
I20260811 02:56:20.834272 4474 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260811 02:56:20.836971 4492 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 02:56:20.838323 4474 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260811 02:56:20.838467 4474 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786416707705494-12843-0/minicluster-data/master-0/data,/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786416707705494-12843-0/minicluster-data/master-0/wal
uuid: "ee79fd986d7b447eb65d8c3fe8dc236a"
format_stamp: "Formatted at 2026-08-11 02:56:20 on dist-test-slave-9gcw"
server_key: "703ef7df7dc005dc6562f34fa3a41549"
server_key_iv: "b4bfcabe537b7ac4d665b594f20512d5"
server_key_version: "encryptionkey@0"
I20260811 02:56:20.838581 4474 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786416707705494-12843-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786416707705494-12843-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786416707705494-12843-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 02:56:20.851756 4474 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 02:56:20.855001 4474 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 02:56:20.855224 4474 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 02:56:20.863323 4474 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:41517
I20260811 02:56:20.863343 4554 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:41517 every 8 connection(s)
I20260811 02:56:20.864447 4474 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786416707705494-12843-0/minicluster-data/master-0/data/info.pb
I20260811 02:56:20.867405 4555 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 02:56:20.869598 12843 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDawEwS/build/debug/bin/kudu as pid 4474
I20260811 02:56:20.869757 12843 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786416707705494-12843-0/minicluster-data/master-0/wal/instance
I20260811 02:56:20.870077 12843 external_mini_cluster.cc:1428] Setting key 5a14ddf557ea2ff64f48d965898e3f63
I20260811 02:56:20.874500 4555 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ee79fd986d7b447eb65d8c3fe8dc236a: Bootstrap starting.
I20260811 02:56:20.876822 4555 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ee79fd986d7b447eb65d8c3fe8dc236a: Neither blocks nor log segments found. Creating new log.
I20260811 02:56:20.877626 4555 log.cc:824] T 00000000000000000000000000000000 P ee79fd986d7b447eb65d8c3fe8dc236a: Log is configured to *not* fsync() on all Append() calls
I20260811 02:56:20.879626 4558 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:56:20.871685 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51661 (local address 127.0.0.1:41517)
0811 02:56:20.872146 (+ 461us) server_negotiation.cc:206] Beginning negotiation
0811 02:56:20.872161 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0811 02:56:20.872197 (+ 36us) server_negotiation.cc:407] Connection header received
0811 02:56:20.872862 (+ 665us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:56:20.872880 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:56:20.873309 (+ 429us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:56:20.873653 (+ 344us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:56:20.874105 (+ 452us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:56:20.875141 (+ 1036us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:56:20.876067 (+ 926us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:56:20.876556 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:56:20.876736 (+ 180us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:56:20.877697 (+ 961us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:56:20.877708 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:56:20.877720 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 02:56:20.878247 (+ 527us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:56:20.878320 (+ 73us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 02:56:20.878357 (+ 37us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:56:20.878465 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0811 02:56:20.878654 (+ 189us) server_negotiation.cc:299] Negotiation successful
0811 02:56:20.878894 (+ 240us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":311,"thread_start_us":142,"threads_started":1}
I20260811 02:56:20.879801 4555 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ee79fd986d7b447eb65d8c3fe8dc236a: No bootstrap required, opened a new log
I20260811 02:56:20.882474 12843 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 02:56:20.883944 4555 raft_consensus.cc:348] T 00000000000000000000000000000000 P ee79fd986d7b447eb65d8c3fe8dc236a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ee79fd986d7b447eb65d8c3fe8dc236a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41517 } }
I20260811 02:56:20.884271 4555 raft_consensus.cc:374] T 00000000000000000000000000000000 P ee79fd986d7b447eb65d8c3fe8dc236a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 02:56:20.884341 4555 raft_consensus.cc:729] T 00000000000000000000000000000000 P ee79fd986d7b447eb65d8c3fe8dc236a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ee79fd986d7b447eb65d8c3fe8dc236a, State: Initialized, Role: FOLLOWER
I20260811 02:56:20.884794 4555 consensus_queue.cc:260] T 00000000000000000000000000000000 P ee79fd986d7b447eb65d8c3fe8dc236a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ee79fd986d7b447eb65d8c3fe8dc236a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41517 } }
I20260811 02:56:20.884963 4555 raft_consensus.cc:388] T 00000000000000000000000000000000 P ee79fd986d7b447eb65d8c3fe8dc236a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 02:56:20.885039 4555 raft_consensus.cc:482] T 00000000000000000000000000000000 P ee79fd986d7b447eb65d8c3fe8dc236a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 02:56:20.885162 4555 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ee79fd986d7b447eb65d8c3fe8dc236a [term 0 FOLLOWER]: Advancing to term 1
I20260811 02:56:20.886330 4555 raft_consensus.cc:504] T 00000000000000000000000000000000 P ee79fd986d7b447eb65d8c3fe8dc236a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ee79fd986d7b447eb65d8c3fe8dc236a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41517 } }
I20260811 02:56:20.886719 4555 leader_election.cc:302] T 00000000000000000000000000000000 P ee79fd986d7b447eb65d8c3fe8dc236a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ee79fd986d7b447eb65d8c3fe8dc236a; no voters:
I20260811 02:56:20.887072 4555 leader_election.cc:288] T 00000000000000000000000000000000 P ee79fd986d7b447eb65d8c3fe8dc236a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 02:56:20.887207 4566 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ee79fd986d7b447eb65d8c3fe8dc236a [term 1 FOLLOWER]: Leader election won for term 1
I20260811 02:56:20.887471 4566 raft_consensus.cc:686] T 00000000000000000000000000000000 P ee79fd986d7b447eb65d8c3fe8dc236a [term 1 LEADER]: Becoming Leader. State: Replica: ee79fd986d7b447eb65d8c3fe8dc236a, State: Running, Role: LEADER
I20260811 02:56:20.887837 4566 consensus_queue.cc:237] T 00000000000000000000000000000000 P ee79fd986d7b447eb65d8c3fe8dc236a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ee79fd986d7b447eb65d8c3fe8dc236a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41517 } }
I20260811 02:56:20.888466 4555 sys_catalog.cc:564] T 00000000000000000000000000000000 P ee79fd986d7b447eb65d8c3fe8dc236a [sys.catalog]: configured and running, proceeding with master startup.
I20260811 02:56:20.889106 4568 sys_catalog.cc:455] T 00000000000000000000000000000000 P ee79fd986d7b447eb65d8c3fe8dc236a [sys.catalog]: SysCatalogTable state changed. Reason: New leader ee79fd986d7b447eb65d8c3fe8dc236a. Latest consensus state: current_term: 1 leader_uuid: "ee79fd986d7b447eb65d8c3fe8dc236a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ee79fd986d7b447eb65d8c3fe8dc236a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41517 } } }
I20260811 02:56:20.889355 4568 sys_catalog.cc:458] T 00000000000000000000000000000000 P ee79fd986d7b447eb65d8c3fe8dc236a [sys.catalog]: This master's current role is: LEADER
I20260811 02:56:20.889861 4558 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:56:20.884918 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41889 (local address 127.0.0.1:41517)
0811 02:56:20.885096 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0811 02:56:20.885100 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 02:56:20.885409 (+ 309us) server_negotiation.cc:407] Connection header received
0811 02:56:20.885543 (+ 134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:56:20.885545 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:56:20.885585 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:56:20.885649 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:56:20.885946 (+ 297us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:56:20.886509 (+ 563us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:56:20.887631 (+ 1122us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:56:20.887968 (+ 337us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:56:20.888049 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:56:20.889107 (+ 1058us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:56:20.889111 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:56:20.889114 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 02:56:20.889125 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:56:20.889182 (+ 57us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 02:56:20.889200 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:56:20.889276 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0811 02:56:20.889654 (+ 378us) server_negotiation.cc:299] Negotiation successful
0811 02:56:20.889711 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":91}
I20260811 02:56:20.890520 4567 sys_catalog.cc:455] T 00000000000000000000000000000000 P ee79fd986d7b447eb65d8c3fe8dc236a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ee79fd986d7b447eb65d8c3fe8dc236a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ee79fd986d7b447eb65d8c3fe8dc236a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41517 } } }
I20260811 02:56:20.890697 4567 sys_catalog.cc:458] T 00000000000000000000000000000000 P ee79fd986d7b447eb65d8c3fe8dc236a [sys.catalog]: This master's current role is: LEADER
W20260811 02:56:20.893465 4584 catalog_manager.cc:1558] T 00000000000000000000000000000000 P ee79fd986d7b447eb65d8c3fe8dc236a: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 02:56:20.893556 4584 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 02:56:20.893641 4578 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 02:56:20.894368 4578 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 02:56:20.900028 4578 catalog_manager.cc:1347] Generated new cluster ID: 4977a88ffa8e4d31bada734152da64c1
I20260811 02:56:20.900095 4578 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 02:56:20.921993 4578 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 02:56:20.923388 4578 catalog_manager.cc:1504] Loading token signing keys...
I20260811 02:56:20.933125 4578 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ee79fd986d7b447eb65d8c3fe8dc236a: Generated new TSK 0
I20260811 02:56:20.934051 4578 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 02:56:20.941977 12843 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDawEwS/build/debug/bin/kudu with pid 4474
2026-08-11T02:56:20Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/4 (1574 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/5
Loading random data
Initializing database '/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786416707705494-12843-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 02:56:21 dist-test-slave-9gcw krb5kdc[4590](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 02:56:21 dist-test-slave-9gcw krb5kdc[4590](info): set up 2 sockets
Aug 11 02:56:21 dist-test-slave-9gcw krb5kdc[4590](info): commencing operation
krb5kdc: starting...
W20260811 02:56:22.090167 12843 mini_kdc.cc:121] Time spent starting KDC: real 1.118s user 0.004s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786416707705494-12843-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786416707705494-12843-0/krb5kdc/test-user.keytab.
Aug 11 02:56:22 dist-test-slave-9gcw krb5kdc[4590](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416982, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T02:56:22Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T02:56:22Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786416707705494-12843-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786416707705494-12843-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260811 02:56:22.229537 12843 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDawEwS/build/debug/bin/kudu
/tmp/dist-test-taskDawEwS/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786416707705494-12843-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786416707705494-12843-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786416707705494-12843-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786416707705494-12843-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45033
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786416707705494-12843-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:38509
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:45033
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786416707705494-12843-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786416707705494-12843-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786416707705494-12843-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 02:56:22.345038 4606 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 02:56:22.345433 4606 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 02:56:22.345526 4606 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 02:56:22.349535 4606 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 02:56:22.349639 4606 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 02:56:22.349676 4606 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 02:56:22.349714 4606 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 02:56:22.349771 4606 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 02:56:22.355188 4606 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:38509
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786416707705494-12843-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786416707705494-12843-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:45033
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786416707705494-12843-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45033
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786416707705494-12843-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4606
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786416707705494-12843-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 02:09:49 UTC on ccycloud.quasar-fsnqqc.root.comops.site
build id 4579570
I20260811 02:56:22.356323 4606 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 02:56:22.357424 4606 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 02:56:22.366067 4613 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:56:22.366128 4615 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:56:22.366097 4612 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:56:22.366351 4606 server_base.cc:1034] running on GCE node
I20260811 02:56:22.366945 4606 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 02:56:22.368206 4606 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 02:56:22.369410 4606 hybrid_clock.cc:648] HybridClock initialized: now 1786416982369384 us; error 52 us; skew 500 ppm
Aug 11 02:56:22 dist-test-slave-9gcw krb5kdc[4590](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416982, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 02:56:22.377730 4606 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260811 02:56:22.378753 4606 webserver.cc:460] Webserver started at http://127.0.0.1:34947/ using document root <none> and password file <none>
I20260811 02:56:22.379390 4606 fs_manager.cc:359] Metadata directory not provided
I20260811 02:56:22.379494 4606 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 02:56:22.379761 4606 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 02:56:22.382041 4606 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786416707705494-12843-0/minicluster-data/master-0/data/instance:
uuid: "0b70b1e56e404855acfe25e98b94c3df"
format_stamp: "Formatted at 2026-08-11 02:56:22 on dist-test-slave-9gcw"
server_key: "28805b0b9eff024530ae0fc0bb9f5f39"
server_key_iv: "f389f47de1177049ad6265c020b33ef2"
server_key_version: "encryptionkey@0"
I20260811 02:56:22.382695 4606 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786416707705494-12843-0/minicluster-data/master-0/wal/instance:
uuid: "0b70b1e56e404855acfe25e98b94c3df"
format_stamp: "Formatted at 2026-08-11 02:56:22 on dist-test-slave-9gcw"
server_key: "28805b0b9eff024530ae0fc0bb9f5f39"
server_key_iv: "f389f47de1177049ad6265c020b33ef2"
server_key_version: "encryptionkey@0"
I20260811 02:56:22.386950 4606 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.001s sys 0.004s
I20260811 02:56:22.389665 4622 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 02:56:22.390941 4606 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260811 02:56:22.391109 4606 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786416707705494-12843-0/minicluster-data/master-0/data,/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786416707705494-12843-0/minicluster-data/master-0/wal
uuid: "0b70b1e56e404855acfe25e98b94c3df"
format_stamp: "Formatted at 2026-08-11 02:56:22 on dist-test-slave-9gcw"
server_key: "28805b0b9eff024530ae0fc0bb9f5f39"
server_key_iv: "f389f47de1177049ad6265c020b33ef2"
server_key_version: "encryptionkey@0"
I20260811 02:56:22.391287 4606 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786416707705494-12843-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786416707705494-12843-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786416707705494-12843-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 02:56:22.396507 4606 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 02:56:22.399829 4606 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 02:56:22.399997 4606 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 02:56:22.407613 4606 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:45033
I20260811 02:56:22.407619 4681 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:45033 every 8 connection(s)
I20260811 02:56:22.408630 4606 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786416707705494-12843-0/minicluster-data/master-0/data/info.pb
I20260811 02:56:22.411430 4682 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 02:56:22.416850 4682 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0b70b1e56e404855acfe25e98b94c3df: Bootstrap starting.
I20260811 02:56:22.417271 12843 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDawEwS/build/debug/bin/kudu as pid 4606
I20260811 02:56:22.417421 12843 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786416707705494-12843-0/minicluster-data/master-0/wal/instance
I20260811 02:56:22.417712 12843 external_mini_cluster.cc:1428] Setting key 02aa7121b4d5286f1a8425ea91b57513
I20260811 02:56:22.419562 4682 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0b70b1e56e404855acfe25e98b94c3df: Neither blocks nor log segments found. Creating new log.
I20260811 02:56:22.420298 4682 log.cc:824] T 00000000000000000000000000000000 P 0b70b1e56e404855acfe25e98b94c3df: Log is configured to *not* fsync() on all Append() calls
I20260811 02:56:22.422585 4682 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0b70b1e56e404855acfe25e98b94c3df: No bootstrap required, opened a new log
I20260811 02:56:22.423413 4685 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:56:22.419080 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38049 (local address 127.0.0.1:45033)
0811 02:56:22.419542 (+ 462us) server_negotiation.cc:206] Beginning negotiation
0811 02:56:22.419558 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0811 02:56:22.419599 (+ 41us) server_negotiation.cc:407] Connection header received
0811 02:56:22.420331 (+ 732us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:56:22.420349 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:56:22.420721 (+ 372us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:56:22.421109 (+ 388us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:56:22.422023 (+ 914us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:56:22.422028 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:56:22.422039 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 02:56:22.422051 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:56:22.422111 (+ 60us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 02:56:22.422142 (+ 31us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:56:22.422243 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0811 02:56:22.422410 (+ 167us) server_negotiation.cc:299] Negotiation successful
0811 02:56:22.422605 (+ 195us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":296,"thread_start_us":154,"threads_started":1}
I20260811 02:56:22.426594 12843 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 02:56:22.426584 4682 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0b70b1e56e404855acfe25e98b94c3df [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0b70b1e56e404855acfe25e98b94c3df" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45033 } }
I20260811 02:56:22.426862 4682 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0b70b1e56e404855acfe25e98b94c3df [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 02:56:22.426908 4682 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0b70b1e56e404855acfe25e98b94c3df [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0b70b1e56e404855acfe25e98b94c3df, State: Initialized, Role: FOLLOWER
I20260811 02:56:22.427361 4682 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0b70b1e56e404855acfe25e98b94c3df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0b70b1e56e404855acfe25e98b94c3df" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45033 } }
I20260811 02:56:22.427531 4682 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0b70b1e56e404855acfe25e98b94c3df [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 02:56:22.427592 4682 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0b70b1e56e404855acfe25e98b94c3df [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 02:56:22.427696 4682 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0b70b1e56e404855acfe25e98b94c3df [term 0 FOLLOWER]: Advancing to term 1
I20260811 02:56:22.428890 4682 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0b70b1e56e404855acfe25e98b94c3df [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0b70b1e56e404855acfe25e98b94c3df" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45033 } }
I20260811 02:56:22.429299 4682 leader_election.cc:302] T 00000000000000000000000000000000 P 0b70b1e56e404855acfe25e98b94c3df [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0b70b1e56e404855acfe25e98b94c3df; no voters:
I20260811 02:56:22.429634 4682 leader_election.cc:288] T 00000000000000000000000000000000 P 0b70b1e56e404855acfe25e98b94c3df [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 02:56:22.429700 4692 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0b70b1e56e404855acfe25e98b94c3df [term 1 FOLLOWER]: Leader election won for term 1
I20260811 02:56:22.429908 4692 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0b70b1e56e404855acfe25e98b94c3df [term 1 LEADER]: Becoming Leader. State: Replica: 0b70b1e56e404855acfe25e98b94c3df, State: Running, Role: LEADER
I20260811 02:56:22.430262 4692 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0b70b1e56e404855acfe25e98b94c3df [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0b70b1e56e404855acfe25e98b94c3df" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45033 } }
I20260811 02:56:22.430934 4682 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0b70b1e56e404855acfe25e98b94c3df [sys.catalog]: configured and running, proceeding with master startup.
I20260811 02:56:22.431125 4685 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:56:22.428962 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58511 (local address 127.0.0.1:45033)
0811 02:56:22.429087 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0811 02:56:22.429091 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 02:56:22.429345 (+ 254us) server_negotiation.cc:407] Connection header received
0811 02:56:22.429508 (+ 163us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:56:22.429512 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:56:22.429555 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:56:22.429641 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:56:22.430538 (+ 897us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:56:22.430545 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:56:22.430547 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 02:56:22.430558 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:56:22.430584 (+ 26us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 02:56:22.430595 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:56:22.430680 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0811 02:56:22.430920 (+ 240us) server_negotiation.cc:299] Negotiation successful
0811 02:56:22.430967 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260811 02:56:22.432103 4693 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0b70b1e56e404855acfe25e98b94c3df [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0b70b1e56e404855acfe25e98b94c3df. Latest consensus state: current_term: 1 leader_uuid: "0b70b1e56e404855acfe25e98b94c3df" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0b70b1e56e404855acfe25e98b94c3df" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45033 } } }
I20260811 02:56:22.432251 4693 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0b70b1e56e404855acfe25e98b94c3df [sys.catalog]: This master's current role is: LEADER
I20260811 02:56:22.432651 4694 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0b70b1e56e404855acfe25e98b94c3df [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0b70b1e56e404855acfe25e98b94c3df" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0b70b1e56e404855acfe25e98b94c3df" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45033 } } }
I20260811 02:56:22.432741 4694 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0b70b1e56e404855acfe25e98b94c3df [sys.catalog]: This master's current role is: LEADER
I20260811 02:56:22.432742 4705 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 02:56:22.436080 4705 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 02:56:22.441483 4705 catalog_manager.cc:1347] Generated new cluster ID: 35ed67124f564cef920d345d60d6cbd9
I20260811 02:56:22.441555 4705 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 02:56:22.462307 4705 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 02:56:22.462669 4705 catalog_manager.cc:1504] Loading token signing keys...
I20260811 02:56:22.467829 4705 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0b70b1e56e404855acfe25e98b94c3df: Generated new TSK 0
I20260811 02:56:22.468508 4705 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 02:56:22.477299 12843 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDawEwS/build/debug/bin/kudu with pid 4606
2026-08-11T02:56:22Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/5 (1529 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/6
Loading random data
Initializing database '/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786416707705494-12843-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 02:56:22 dist-test-slave-9gcw krb5kdc[4720](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 02:56:22 dist-test-slave-9gcw krb5kdc[4720](info): set up 2 sockets
Aug 11 02:56:22 dist-test-slave-9gcw krb5kdc[4720](info): commencing operation
krb5kdc: starting...
W20260811 02:56:23.621546 12843 mini_kdc.cc:121] Time spent starting KDC: real 1.120s user 0.006s sys 0.002s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786416707705494-12843-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786416707705494-12843-0/krb5kdc/test-user.keytab.
Aug 11 02:56:23 dist-test-slave-9gcw krb5kdc[4720](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416983, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T02:56:23Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T02:56:23Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786416707705494-12843-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786416707705494-12843-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260811 02:56:23.760914 12843 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDawEwS/build/debug/bin/kudu
/tmp/dist-test-taskDawEwS/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786416707705494-12843-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786416707705494-12843-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786416707705494-12843-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786416707705494-12843-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33745
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786416707705494-12843-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:38887
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:33745
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786416707705494-12843-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786416707705494-12843-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786416707705494-12843-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 02:56:23.880239 4735 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 02:56:23.880615 4735 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 02:56:23.880764 4735 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 02:56:23.884934 4735 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 02:56:23.885046 4735 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 02:56:23.885087 4735 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 02:56:23.885125 4735 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 02:56:23.885191 4735 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 02:56:23.890899 4735 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:38887
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786416707705494-12843-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786416707705494-12843-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:33745
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786416707705494-12843-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33745
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786416707705494-12843-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4735
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786416707705494-12843-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 02:09:49 UTC on ccycloud.quasar-fsnqqc.root.comops.site
build id 4579570
I20260811 02:56:23.892087 4735 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 02:56:23.893272 4735 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 02:56:23.902498 4742 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:56:23.902498 4744 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:56:23.902544 4741 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:56:23.902958 4735 server_base.cc:1034] running on GCE node
I20260811 02:56:23.903631 4735 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 02:56:23.904721 4735 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 02:56:23.905887 4735 hybrid_clock.cc:648] HybridClock initialized: now 1786416983905861 us; error 46 us; skew 500 ppm
Aug 11 02:56:23 dist-test-slave-9gcw krb5kdc[4720](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416983, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 02:56:23.908890 4735 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260811 02:56:23.909884 4735 webserver.cc:460] Webserver started at http://127.0.0.1:40701/ using document root <none> and password file <none>
I20260811 02:56:23.910434 4735 fs_manager.cc:359] Metadata directory not provided
I20260811 02:56:23.910494 4735 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 02:56:23.910682 4735 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 02:56:23.912767 4735 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786416707705494-12843-0/minicluster-data/master-0/data/instance:
uuid: "7ef9e72a1ff544ba9637b5b9c6a148c7"
format_stamp: "Formatted at 2026-08-11 02:56:23 on dist-test-slave-9gcw"
server_key: "08b1cae7096ceb923e69fd70312e51a2"
server_key_iv: "9256d8199af7604e384623f62751900d"
server_key_version: "encryptionkey@0"
I20260811 02:56:23.913378 4735 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786416707705494-12843-0/minicluster-data/master-0/wal/instance:
uuid: "7ef9e72a1ff544ba9637b5b9c6a148c7"
format_stamp: "Formatted at 2026-08-11 02:56:23 on dist-test-slave-9gcw"
server_key: "08b1cae7096ceb923e69fd70312e51a2"
server_key_iv: "9256d8199af7604e384623f62751900d"
server_key_version: "encryptionkey@0"
I20260811 02:56:23.917392 4735 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260811 02:56:23.919975 4753 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 02:56:23.921674 4735 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 02:56:23.921784 4735 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786416707705494-12843-0/minicluster-data/master-0/data,/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786416707705494-12843-0/minicluster-data/master-0/wal
uuid: "7ef9e72a1ff544ba9637b5b9c6a148c7"
format_stamp: "Formatted at 2026-08-11 02:56:23 on dist-test-slave-9gcw"
server_key: "08b1cae7096ceb923e69fd70312e51a2"
server_key_iv: "9256d8199af7604e384623f62751900d"
server_key_version: "encryptionkey@0"
I20260811 02:56:23.921900 4735 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786416707705494-12843-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786416707705494-12843-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786416707705494-12843-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 02:56:23.947297 4735 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 02:56:23.950508 4735 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 02:56:23.950680 4735 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 02:56:23.958662 4735 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:33745
I20260811 02:56:23.958690 4814 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:33745 every 8 connection(s)
I20260811 02:56:23.959704 4735 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786416707705494-12843-0/minicluster-data/master-0/data/info.pb
I20260811 02:56:23.962672 4815 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 02:56:23.968393 4815 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7ef9e72a1ff544ba9637b5b9c6a148c7: Bootstrap starting.
I20260811 02:56:23.969043 12843 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDawEwS/build/debug/bin/kudu as pid 4735
I20260811 02:56:23.969214 12843 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786416707705494-12843-0/minicluster-data/master-0/wal/instance
I20260811 02:56:23.969523 12843 external_mini_cluster.cc:1428] Setting key 229be0cd2346c1b81443d75a1b047b88
I20260811 02:56:23.971797 4815 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7ef9e72a1ff544ba9637b5b9c6a148c7: Neither blocks nor log segments found. Creating new log.
I20260811 02:56:23.972565 4815 log.cc:824] T 00000000000000000000000000000000 P 7ef9e72a1ff544ba9637b5b9c6a148c7: Log is configured to *not* fsync() on all Append() calls
I20260811 02:56:23.974896 4815 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7ef9e72a1ff544ba9637b5b9c6a148c7: No bootstrap required, opened a new log
I20260811 02:56:23.979857 4815 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7ef9e72a1ff544ba9637b5b9c6a148c7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7ef9e72a1ff544ba9637b5b9c6a148c7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33745 } }
I20260811 02:56:23.980139 4815 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7ef9e72a1ff544ba9637b5b9c6a148c7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 02:56:23.980218 4815 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7ef9e72a1ff544ba9637b5b9c6a148c7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7ef9e72a1ff544ba9637b5b9c6a148c7, State: Initialized, Role: FOLLOWER
I20260811 02:56:23.980726 4815 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7ef9e72a1ff544ba9637b5b9c6a148c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7ef9e72a1ff544ba9637b5b9c6a148c7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33745 } }
I20260811 02:56:23.980888 4815 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7ef9e72a1ff544ba9637b5b9c6a148c7 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 02:56:23.980964 4815 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7ef9e72a1ff544ba9637b5b9c6a148c7 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 02:56:23.981069 4815 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7ef9e72a1ff544ba9637b5b9c6a148c7 [term 0 FOLLOWER]: Advancing to term 1
Aug 11 02:56:23 dist-test-slave-9gcw krb5kdc[4720](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416983, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260811 02:56:23.982261 4815 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7ef9e72a1ff544ba9637b5b9c6a148c7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7ef9e72a1ff544ba9637b5b9c6a148c7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33745 } }
I20260811 02:56:23.982690 4815 leader_election.cc:302] T 00000000000000000000000000000000 P 7ef9e72a1ff544ba9637b5b9c6a148c7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7ef9e72a1ff544ba9637b5b9c6a148c7; no voters:
I20260811 02:56:23.983063 4815 leader_election.cc:288] T 00000000000000000000000000000000 P 7ef9e72a1ff544ba9637b5b9c6a148c7 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 02:56:23.983260 4822 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7ef9e72a1ff544ba9637b5b9c6a148c7 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 02:56:23.983922 4822 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7ef9e72a1ff544ba9637b5b9c6a148c7 [term 1 LEADER]: Becoming Leader. State: Replica: 7ef9e72a1ff544ba9637b5b9c6a148c7, State: Running, Role: LEADER
I20260811 02:56:23.984277 4822 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7ef9e72a1ff544ba9637b5b9c6a148c7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7ef9e72a1ff544ba9637b5b9c6a148c7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33745 } }
I20260811 02:56:23.984598 4815 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7ef9e72a1ff544ba9637b5b9c6a148c7 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 02:56:23.986415 4823 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7ef9e72a1ff544ba9637b5b9c6a148c7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7ef9e72a1ff544ba9637b5b9c6a148c7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7ef9e72a1ff544ba9637b5b9c6a148c7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33745 } } }
I20260811 02:56:23.986414 4824 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7ef9e72a1ff544ba9637b5b9c6a148c7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7ef9e72a1ff544ba9637b5b9c6a148c7. Latest consensus state: current_term: 1 leader_uuid: "7ef9e72a1ff544ba9637b5b9c6a148c7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7ef9e72a1ff544ba9637b5b9c6a148c7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33745 } } }
I20260811 02:56:23.986590 4823 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7ef9e72a1ff544ba9637b5b9c6a148c7 [sys.catalog]: This master's current role is: LEADER
I20260811 02:56:23.986590 4824 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7ef9e72a1ff544ba9637b5b9c6a148c7 [sys.catalog]: This master's current role is: LEADER
I20260811 02:56:23.988639 4819 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:56:23.971202 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.180:44723 (local address 127.0.0.1:33745)
0811 02:56:23.971809 (+ 607us) server_negotiation.cc:206] Beginning negotiation
0811 02:56:23.971828 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0811 02:56:23.971903 (+ 75us) server_negotiation.cc:407] Connection header received
0811 02:56:23.972708 (+ 805us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:56:23.972740 (+ 32us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:56:23.973247 (+ 507us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:56:23.973717 (+ 470us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:56:23.974670 (+ 953us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:56:23.976138 (+ 1468us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:56:23.977555 (+ 1417us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:56:23.978428 (+ 873us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:56:23.978619 (+ 191us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:56:23.982153 (+ 3534us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:56:23.982162 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:56:23.982194 (+ 32us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:56:23.982237 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:56:23.984610 (+ 2373us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:56:23.985417 (+ 807us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:56:23.985424 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:56:23.985433 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:56:23.985606 (+ 173us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:56:23.986091 (+ 485us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:56:23.986094 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:56:23.986096 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:56:23.986698 (+ 602us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:56:23.986998 (+ 300us) server_negotiation.cc:1035] Waiting for connection context
0811 02:56:23.987424 (+ 426us) server_negotiation.cc:299] Negotiation successful
0811 02:56:23.987654 (+ 230us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":378,"thread_start_us":186,"threads_started":1}
W20260811 02:56:23.990082 4837 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 7ef9e72a1ff544ba9637b5b9c6a148c7: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 02:56:23.990175 4837 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 02:56:23.990447 4836 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 02:56:23.991530 4836 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 02:56:23.992312 12843 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 02:56:23.998225 4836 catalog_manager.cc:1347] Generated new cluster ID: 8ff59c465eae44cabc5d1d9c8030febc
I20260811 02:56:23.998309 4836 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 02:56:24.007704 4819 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:56:23.994837 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.180:59247 (local address 127.0.0.1:33745)
0811 02:56:23.994979 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0811 02:56:23.994985 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 02:56:23.997507 (+ 2522us) server_negotiation.cc:407] Connection header received
0811 02:56:23.997750 (+ 243us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:56:23.997754 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:56:23.997818 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:56:23.997920 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:56:23.998974 (+ 1054us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:56:23.999865 (+ 891us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:56:24.001098 (+ 1233us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:56:24.001512 (+ 414us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:56:24.001593 (+ 81us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:56:24.003664 (+ 2071us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:56:24.003668 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:56:24.003671 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:56:24.003702 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:56:24.005507 (+ 1805us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:56:24.006125 (+ 618us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:56:24.006129 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:56:24.006131 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:56:24.006235 (+ 104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:56:24.006633 (+ 398us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:56:24.006637 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:56:24.006639 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:56:24.006854 (+ 215us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:56:24.006969 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0811 02:56:24.007423 (+ 454us) server_negotiation.cc:299] Negotiation successful
0811 02:56:24.007532 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260811 02:56:24.009341 4836 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 02:56:24.010332 4836 catalog_manager.cc:1504] Loading token signing keys...
I20260811 02:56:24.032389 4836 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7ef9e72a1ff544ba9637b5b9c6a148c7: Generated new TSK 0
I20260811 02:56:24.033560 4836 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 02:56:24.037902 12843 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDawEwS/build/debug/bin/kudu with pid 4735
W20260811 02:56:24.046569 4718 connection.cc:535] client connection to 127.0.0.1:33745 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:33745): Connection reset by peer (error 104)
2026-08-11T02:56:24Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/6 (1559 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/7
Loading random data
Initializing database '/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786416707705494-12843-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 02:56:24 dist-test-slave-9gcw krb5kdc[4848](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 02:56:24 dist-test-slave-9gcw krb5kdc[4848](info): set up 2 sockets
Aug 11 02:56:24 dist-test-slave-9gcw krb5kdc[4848](info): commencing operation
krb5kdc: starting...
W20260811 02:56:25.290074 12843 mini_kdc.cc:121] Time spent starting KDC: real 1.229s user 0.002s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786416707705494-12843-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786416707705494-12843-0/krb5kdc/test-user.keytab.
Aug 11 02:56:25 dist-test-slave-9gcw krb5kdc[4848](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416985, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T02:56:25Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T02:56:25Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786416707705494-12843-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786416707705494-12843-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260811 02:56:25.428931 12843 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDawEwS/build/debug/bin/kudu
/tmp/dist-test-taskDawEwS/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786416707705494-12843-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786416707705494-12843-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786416707705494-12843-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786416707705494-12843-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43925
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786416707705494-12843-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:35115
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:43925
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786416707705494-12843-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786416707705494-12843-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786416707705494-12843-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 02:56:25.550858 4865 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 02:56:25.551159 4865 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 02:56:25.551206 4865 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 02:56:25.555295 4865 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 02:56:25.555392 4865 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 02:56:25.555415 4865 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 02:56:25.555435 4865 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 02:56:25.555505 4865 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 02:56:25.561072 4865 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:35115
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786416707705494-12843-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786416707705494-12843-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:43925
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786416707705494-12843-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43925
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786416707705494-12843-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4865
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786416707705494-12843-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 02:09:49 UTC on ccycloud.quasar-fsnqqc.root.comops.site
build id 4579570
I20260811 02:56:25.562232 4865 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 02:56:25.563297 4865 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 02:56:25.572552 4871 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:56:25.572638 4870 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:56:25.572794 4865 server_base.cc:1034] running on GCE node
W20260811 02:56:25.572564 4873 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:56:25.573580 4865 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 02:56:25.574671 4865 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 02:56:25.575892 4865 hybrid_clock.cc:648] HybridClock initialized: now 1786416985575852 us; error 54 us; skew 500 ppm
Aug 11 02:56:25 dist-test-slave-9gcw krb5kdc[4848](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416985, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 02:56:25.578832 4865 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260811 02:56:25.579782 4865 webserver.cc:460] Webserver started at http://127.0.0.1:40423/ using document root <none> and password file <none>
I20260811 02:56:25.580376 4865 fs_manager.cc:359] Metadata directory not provided
I20260811 02:56:25.580472 4865 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 02:56:25.580716 4865 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 02:56:25.582839 4865 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786416707705494-12843-0/minicluster-data/master-0/data/instance:
uuid: "3b15e86d4bf5449bb1d1984a20d54342"
format_stamp: "Formatted at 2026-08-11 02:56:25 on dist-test-slave-9gcw"
server_key: "f28e32a092d6670893ed7ce65d2bf264"
server_key_iv: "0b6b1559a9e0d10010b4034cf1d71253"
server_key_version: "encryptionkey@0"
I20260811 02:56:25.583427 4865 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786416707705494-12843-0/minicluster-data/master-0/wal/instance:
uuid: "3b15e86d4bf5449bb1d1984a20d54342"
format_stamp: "Formatted at 2026-08-11 02:56:25 on dist-test-slave-9gcw"
server_key: "f28e32a092d6670893ed7ce65d2bf264"
server_key_iv: "0b6b1559a9e0d10010b4034cf1d71253"
server_key_version: "encryptionkey@0"
I20260811 02:56:25.587535 4865 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.001s
I20260811 02:56:25.590044 4881 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 02:56:25.591336 4865 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.003s
I20260811 02:56:25.591478 4865 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786416707705494-12843-0/minicluster-data/master-0/data,/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786416707705494-12843-0/minicluster-data/master-0/wal
uuid: "3b15e86d4bf5449bb1d1984a20d54342"
format_stamp: "Formatted at 2026-08-11 02:56:25 on dist-test-slave-9gcw"
server_key: "f28e32a092d6670893ed7ce65d2bf264"
server_key_iv: "0b6b1559a9e0d10010b4034cf1d71253"
server_key_version: "encryptionkey@0"
I20260811 02:56:25.591594 4865 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786416707705494-12843-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786416707705494-12843-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786416707705494-12843-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 02:56:25.609431 4865 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 02:56:25.612885 4865 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 02:56:25.613121 4865 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 02:56:25.620599 4865 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:43925
I20260811 02:56:25.620620 4940 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:43925 every 8 connection(s)
I20260811 02:56:25.621661 4865 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786416707705494-12843-0/minicluster-data/master-0/data/info.pb
I20260811 02:56:25.624537 4942 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 02:56:25.627183 12843 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDawEwS/build/debug/bin/kudu as pid 4865
I20260811 02:56:25.627360 12843 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786416707705494-12843-0/minicluster-data/master-0/wal/instance
I20260811 02:56:25.627656 12843 external_mini_cluster.cc:1428] Setting key d8a4188ab8fc4d22b9c756cc7701d84e
I20260811 02:56:25.631457 4942 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3b15e86d4bf5449bb1d1984a20d54342: Bootstrap starting.
I20260811 02:56:25.634254 4942 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3b15e86d4bf5449bb1d1984a20d54342: Neither blocks nor log segments found. Creating new log.
I20260811 02:56:25.635215 4942 log.cc:824] T 00000000000000000000000000000000 P 3b15e86d4bf5449bb1d1984a20d54342: Log is configured to *not* fsync() on all Append() calls
I20260811 02:56:25.637995 4942 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3b15e86d4bf5449bb1d1984a20d54342: No bootstrap required, opened a new log
I20260811 02:56:25.639024 4950 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:56:25.629374 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.180:35025 (local address 127.0.0.1:43925)
0811 02:56:25.629840 (+ 466us) server_negotiation.cc:206] Beginning negotiation
0811 02:56:25.629858 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0811 02:56:25.629901 (+ 43us) server_negotiation.cc:407] Connection header received
0811 02:56:25.630762 (+ 861us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:56:25.630787 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:56:25.631230 (+ 443us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:56:25.631607 (+ 377us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:56:25.632042 (+ 435us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:56:25.633557 (+ 1515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:56:25.634787 (+ 1230us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:56:25.635535 (+ 748us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:56:25.635719 (+ 184us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:56:25.636906 (+ 1187us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:56:25.636913 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:56:25.636929 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 02:56:25.637479 (+ 550us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:56:25.637550 (+ 71us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 02:56:25.637580 (+ 30us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:56:25.637755 (+ 175us) server_negotiation.cc:1035] Waiting for connection context
0811 02:56:25.637927 (+ 172us) server_negotiation.cc:299] Negotiation successful
0811 02:56:25.638069 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":312,"thread_start_us":141,"threads_started":1}
I20260811 02:56:25.642333 12843 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 02:56:25.642463 4942 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3b15e86d4bf5449bb1d1984a20d54342 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3b15e86d4bf5449bb1d1984a20d54342" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43925 } }
I20260811 02:56:25.642735 4942 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3b15e86d4bf5449bb1d1984a20d54342 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 02:56:25.642813 4942 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3b15e86d4bf5449bb1d1984a20d54342 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3b15e86d4bf5449bb1d1984a20d54342, State: Initialized, Role: FOLLOWER
I20260811 02:56:25.643298 4942 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3b15e86d4bf5449bb1d1984a20d54342 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3b15e86d4bf5449bb1d1984a20d54342" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43925 } }
I20260811 02:56:25.643455 4942 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3b15e86d4bf5449bb1d1984a20d54342 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 02:56:25.643543 4942 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3b15e86d4bf5449bb1d1984a20d54342 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 02:56:25.643643 4942 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3b15e86d4bf5449bb1d1984a20d54342 [term 0 FOLLOWER]: Advancing to term 1
I20260811 02:56:25.644807 4942 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3b15e86d4bf5449bb1d1984a20d54342 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3b15e86d4bf5449bb1d1984a20d54342" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43925 } }
I20260811 02:56:25.645291 4942 leader_election.cc:302] T 00000000000000000000000000000000 P 3b15e86d4bf5449bb1d1984a20d54342 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3b15e86d4bf5449bb1d1984a20d54342; no voters:
I20260811 02:56:25.645668 4942 leader_election.cc:288] T 00000000000000000000000000000000 P 3b15e86d4bf5449bb1d1984a20d54342 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 02:56:25.645920 4961 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3b15e86d4bf5449bb1d1984a20d54342 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 02:56:25.646186 4961 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3b15e86d4bf5449bb1d1984a20d54342 [term 1 LEADER]: Becoming Leader. State: Replica: 3b15e86d4bf5449bb1d1984a20d54342, State: Running, Role: LEADER
I20260811 02:56:25.646615 4961 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3b15e86d4bf5449bb1d1984a20d54342 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3b15e86d4bf5449bb1d1984a20d54342" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43925 } }
I20260811 02:56:25.647181 4942 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3b15e86d4bf5449bb1d1984a20d54342 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 02:56:25.648452 4963 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3b15e86d4bf5449bb1d1984a20d54342 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3b15e86d4bf5449bb1d1984a20d54342" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3b15e86d4bf5449bb1d1984a20d54342" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43925 } } }
I20260811 02:56:25.648594 4963 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3b15e86d4bf5449bb1d1984a20d54342 [sys.catalog]: This master's current role is: LEADER
I20260811 02:56:25.648869 4963 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3b15e86d4bf5449bb1d1984a20d54342 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3b15e86d4bf5449bb1d1984a20d54342. Latest consensus state: current_term: 1 leader_uuid: "3b15e86d4bf5449bb1d1984a20d54342" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3b15e86d4bf5449bb1d1984a20d54342" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43925 } } }
I20260811 02:56:25.648952 4963 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3b15e86d4bf5449bb1d1984a20d54342 [sys.catalog]: This master's current role is: LEADER
I20260811 02:56:25.650516 4950 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:56:25.644888 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.180:35415 (local address 127.0.0.1:43925)
0811 02:56:25.645079 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0811 02:56:25.645084 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 02:56:25.645179 (+ 95us) server_negotiation.cc:407] Connection header received
0811 02:56:25.645356 (+ 177us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:56:25.645359 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:56:25.645412 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:56:25.645493 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:56:25.645967 (+ 474us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:56:25.646853 (+ 886us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:56:25.648115 (+ 1262us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:56:25.648457 (+ 342us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:56:25.648555 (+ 98us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:56:25.649770 (+ 1215us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:56:25.649778 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:56:25.649784 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 02:56:25.649803 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:56:25.649834 (+ 31us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 02:56:25.649849 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:56:25.649952 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0811 02:56:25.650340 (+ 388us) server_negotiation.cc:299] Negotiation successful
0811 02:56:25.650377 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":94}
W20260811 02:56:25.652823 4978 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 3b15e86d4bf5449bb1d1984a20d54342: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 02:56:25.652896 4978 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 02:56:25.652983 4974 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 02:56:25.653811 4974 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 02:56:25.659894 4974 catalog_manager.cc:1347] Generated new cluster ID: 85d30f7bb78a4bb4883b1453a90b9c09
I20260811 02:56:25.659972 4974 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 02:56:25.666023 4974 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 02:56:25.667289 4974 catalog_manager.cc:1504] Loading token signing keys...
I20260811 02:56:25.676221 4974 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3b15e86d4bf5449bb1d1984a20d54342: Generated new TSK 0
I20260811 02:56:25.677027 4974 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 02:56:25.682991 12843 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDawEwS/build/debug/bin/kudu with pid 4865
W20260811 02:56:25.692298 4846 connection.cc:535] client connection to 127.0.0.1:43925 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:43925): Connection reset by peer (error 104)
2026-08-11T02:56:25Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/7 (1645 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/8
Loading random data
Initializing database '/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786416707705494-12843-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 02:56:25 dist-test-slave-9gcw krb5kdc[4984](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 02:56:25 dist-test-slave-9gcw krb5kdc[4984](info): set up 2 sockets
Aug 11 02:56:25 dist-test-slave-9gcw krb5kdc[4984](info): commencing operation
krb5kdc: starting...
W20260811 02:56:26.837819 12843 mini_kdc.cc:121] Time spent starting KDC: real 1.131s user 0.003s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786416707705494-12843-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786416707705494-12843-0/krb5kdc/test-user.keytab.
Aug 11 02:56:26 dist-test-slave-9gcw krb5kdc[4984](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416986, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T02:56:26Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T02:56:26Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786416707705494-12843-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786416707705494-12843-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260811 02:56:26.981011 12843 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDawEwS/build/debug/bin/kudu
/tmp/dist-test-taskDawEwS/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786416707705494-12843-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786416707705494-12843-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786416707705494-12843-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786416707705494-12843-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45087
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786416707705494-12843-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:33525
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:45087
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786416707705494-12843-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786416707705494-12843-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786416707705494-12843-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 02:56:27.113054 4997 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 02:56:27.113397 4997 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 02:56:27.113440 4997 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 02:56:27.117573 4997 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 02:56:27.117662 4997 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 02:56:27.117684 4997 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 02:56:27.117705 4997 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 02:56:27.117720 4997 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 02:56:27.123239 4997 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:33525
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786416707705494-12843-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786416707705494-12843-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:45087
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786416707705494-12843-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45087
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786416707705494-12843-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4997
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786416707705494-12843-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 02:09:49 UTC on ccycloud.quasar-fsnqqc.root.comops.site
build id 4579570
I20260811 02:56:27.124368 4997 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 02:56:27.125433 4997 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260811 02:56:27.134537 4997 server_base.cc:1034] running on GCE node
W20260811 02:56:27.134716 5002 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:56:27.134785 5005 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:56:27.134819 5003 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:56:27.135510 4997 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 02:56:27.136739 4997 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 02:56:27.137974 4997 hybrid_clock.cc:648] HybridClock initialized: now 1786416987137878 us; error 106 us; skew 500 ppm
Aug 11 02:56:27 dist-test-slave-9gcw krb5kdc[4984](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416987, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 02:56:27.141487 4997 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260811 02:56:27.142522 4997 webserver.cc:460] Webserver started at http://127.0.0.1:44269/ using document root <none> and password file <none>
I20260811 02:56:27.143153 4997 fs_manager.cc:359] Metadata directory not provided
I20260811 02:56:27.143225 4997 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 02:56:27.143430 4997 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 02:56:27.145843 4997 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786416707705494-12843-0/minicluster-data/master-0/data/instance:
uuid: "8ca8a344391648279d436936be95dc7d"
format_stamp: "Formatted at 2026-08-11 02:56:27 on dist-test-slave-9gcw"
server_key: "539faacaab3c020b91ba5d5efcdd3db5"
server_key_iv: "6ee83e34e54abb7d9920abb962b781fd"
server_key_version: "encryptionkey@0"
I20260811 02:56:27.146478 4997 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786416707705494-12843-0/minicluster-data/master-0/wal/instance:
uuid: "8ca8a344391648279d436936be95dc7d"
format_stamp: "Formatted at 2026-08-11 02:56:27 on dist-test-slave-9gcw"
server_key: "539faacaab3c020b91ba5d5efcdd3db5"
server_key_iv: "6ee83e34e54abb7d9920abb962b781fd"
server_key_version: "encryptionkey@0"
I20260811 02:56:27.151036 4997 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260811 02:56:27.154042 5013 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 02:56:27.155686 4997 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 02:56:27.155817 4997 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786416707705494-12843-0/minicluster-data/master-0/data,/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786416707705494-12843-0/minicluster-data/master-0/wal
uuid: "8ca8a344391648279d436936be95dc7d"
format_stamp: "Formatted at 2026-08-11 02:56:27 on dist-test-slave-9gcw"
server_key: "539faacaab3c020b91ba5d5efcdd3db5"
server_key_iv: "6ee83e34e54abb7d9920abb962b781fd"
server_key_version: "encryptionkey@0"
I20260811 02:56:27.155925 4997 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786416707705494-12843-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786416707705494-12843-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786416707705494-12843-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 02:56:27.160825 4997 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 02:56:27.164394 4997 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 02:56:27.164559 4997 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 02:56:27.172433 4997 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:45087
I20260811 02:56:27.172462 5072 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:45087 every 8 connection(s)
I20260811 02:56:27.173488 4997 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786416707705494-12843-0/minicluster-data/master-0/data/info.pb
I20260811 02:56:27.176889 5073 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 02:56:27.179216 12843 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDawEwS/build/debug/bin/kudu as pid 4997
I20260811 02:56:27.179383 12843 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786416707705494-12843-0/minicluster-data/master-0/wal/instance
I20260811 02:56:27.179688 12843 external_mini_cluster.cc:1428] Setting key 79b580e081162821bb907774d6f7179f
I20260811 02:56:27.184623 5073 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8ca8a344391648279d436936be95dc7d: Bootstrap starting.
I20260811 02:56:27.185423 5076 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:56:27.181259 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.180:44381 (local address 127.0.0.1:45087)
0811 02:56:27.181752 (+ 493us) server_negotiation.cc:206] Beginning negotiation
0811 02:56:27.181771 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0811 02:56:27.181810 (+ 39us) server_negotiation.cc:407] Connection header received
0811 02:56:27.182483 (+ 673us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:56:27.182501 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:56:27.182810 (+ 309us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:56:27.183165 (+ 355us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:56:27.184057 (+ 892us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:56:27.184062 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:56:27.184074 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 02:56:27.184085 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:56:27.184142 (+ 57us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 02:56:27.184162 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:56:27.184348 (+ 186us) server_negotiation.cc:1035] Waiting for connection context
0811 02:56:27.184423 (+ 75us) server_negotiation.cc:299] Negotiation successful
0811 02:56:27.184586 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":344,"thread_start_us":173,"threads_started":1}
I20260811 02:56:27.187567 5073 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8ca8a344391648279d436936be95dc7d: Neither blocks nor log segments found. Creating new log.
I20260811 02:56:27.188479 12843 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 02:56:27.188639 5073 log.cc:824] T 00000000000000000000000000000000 P 8ca8a344391648279d436936be95dc7d: Log is configured to *not* fsync() on all Append() calls
I20260811 02:56:27.191334 5073 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8ca8a344391648279d436936be95dc7d: No bootstrap required, opened a new log
I20260811 02:56:27.193131 5076 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:56:27.190950 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.180:56281 (local address 127.0.0.1:45087)
0811 02:56:27.191084 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0811 02:56:27.191088 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 02:56:27.191349 (+ 261us) server_negotiation.cc:407] Connection header received
0811 02:56:27.191557 (+ 208us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:56:27.191561 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:56:27.191604 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:56:27.191712 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:56:27.192640 (+ 928us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:56:27.192644 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:56:27.192647 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 02:56:27.192662 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:56:27.192725 (+ 63us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 02:56:27.192744 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:56:27.192856 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0811 02:56:27.192926 (+ 70us) server_negotiation.cc:299] Negotiation successful
0811 02:56:27.192971 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260811 02:56:27.196132 5073 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8ca8a344391648279d436936be95dc7d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8ca8a344391648279d436936be95dc7d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45087 } }
I20260811 02:56:27.196441 5073 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8ca8a344391648279d436936be95dc7d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 02:56:27.196506 5073 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8ca8a344391648279d436936be95dc7d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8ca8a344391648279d436936be95dc7d, State: Initialized, Role: FOLLOWER
I20260811 02:56:27.196986 5073 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8ca8a344391648279d436936be95dc7d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8ca8a344391648279d436936be95dc7d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45087 } }
I20260811 02:56:27.197196 5073 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8ca8a344391648279d436936be95dc7d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 02:56:27.197281 5073 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8ca8a344391648279d436936be95dc7d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 02:56:27.197371 5073 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8ca8a344391648279d436936be95dc7d [term 0 FOLLOWER]: Advancing to term 1
I20260811 02:56:27.198678 5073 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8ca8a344391648279d436936be95dc7d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8ca8a344391648279d436936be95dc7d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45087 } }
I20260811 02:56:27.199012 5073 leader_election.cc:302] T 00000000000000000000000000000000 P 8ca8a344391648279d436936be95dc7d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8ca8a344391648279d436936be95dc7d; no voters:
I20260811 02:56:27.199319 5073 leader_election.cc:288] T 00000000000000000000000000000000 P 8ca8a344391648279d436936be95dc7d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 02:56:27.199440 5085 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8ca8a344391648279d436936be95dc7d [term 1 FOLLOWER]: Leader election won for term 1
I20260811 02:56:27.199718 5085 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8ca8a344391648279d436936be95dc7d [term 1 LEADER]: Becoming Leader. State: Replica: 8ca8a344391648279d436936be95dc7d, State: Running, Role: LEADER
I20260811 02:56:27.200066 5085 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8ca8a344391648279d436936be95dc7d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8ca8a344391648279d436936be95dc7d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45087 } }
I20260811 02:56:27.200896 5073 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8ca8a344391648279d436936be95dc7d [sys.catalog]: configured and running, proceeding with master startup.
I20260811 02:56:27.201898 5086 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8ca8a344391648279d436936be95dc7d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8ca8a344391648279d436936be95dc7d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8ca8a344391648279d436936be95dc7d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45087 } } }
I20260811 02:56:27.202039 5086 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8ca8a344391648279d436936be95dc7d [sys.catalog]: This master's current role is: LEADER
I20260811 02:56:27.201967 5087 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8ca8a344391648279d436936be95dc7d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8ca8a344391648279d436936be95dc7d. Latest consensus state: current_term: 1 leader_uuid: "8ca8a344391648279d436936be95dc7d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8ca8a344391648279d436936be95dc7d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45087 } } }
I20260811 02:56:27.202131 5087 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8ca8a344391648279d436936be95dc7d [sys.catalog]: This master's current role is: LEADER
I20260811 02:56:27.203496 5092 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 02:56:27.206379 5092 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 02:56:27.212365 5092 catalog_manager.cc:1347] Generated new cluster ID: 0b9fc71840384b459ef870d583505c5a
I20260811 02:56:27.212457 5092 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 02:56:27.224596 5092 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 02:56:27.225286 5092 catalog_manager.cc:1504] Loading token signing keys...
I20260811 02:56:27.232069 5092 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8ca8a344391648279d436936be95dc7d: Generated new TSK 0
I20260811 02:56:27.232867 5092 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 02:56:27.236352 12843 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDawEwS/build/debug/bin/kudu with pid 4997
2026-08-11T02:56:27Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/8 (1553 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/9
Loading random data
Initializing database '/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786416707705494-12843-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 02:56:27 dist-test-slave-9gcw krb5kdc[5108](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 02:56:27 dist-test-slave-9gcw krb5kdc[5108](info): set up 2 sockets
Aug 11 02:56:27 dist-test-slave-9gcw krb5kdc[5108](info): commencing operation
krb5kdc: starting...
W20260811 02:56:28.379403 12843 mini_kdc.cc:121] Time spent starting KDC: real 1.118s user 0.002s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786416707705494-12843-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786416707705494-12843-0/krb5kdc/test-user.keytab.
Aug 11 02:56:28 dist-test-slave-9gcw krb5kdc[5108](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416988, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T02:56:28Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T02:56:28Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786416707705494-12843-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786416707705494-12843-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260811 02:56:28.522312 12843 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDawEwS/build/debug/bin/kudu
/tmp/dist-test-taskDawEwS/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786416707705494-12843-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786416707705494-12843-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786416707705494-12843-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786416707705494-12843-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45165
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786416707705494-12843-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:34255
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:45165
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786416707705494-12843-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786416707705494-12843-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786416707705494-12843-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 02:56:28.638331 5122 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 02:56:28.638708 5122 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 02:56:28.638793 5122 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 02:56:28.642956 5122 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 02:56:28.643056 5122 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 02:56:28.643096 5122 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 02:56:28.643136 5122 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 02:56:28.643172 5122 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 02:56:28.648658 5122 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:34255
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786416707705494-12843-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786416707705494-12843-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:45165
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786416707705494-12843-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45165
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786416707705494-12843-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5122
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786416707705494-12843-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 02:09:49 UTC on ccycloud.quasar-fsnqqc.root.comops.site
build id 4579570
I20260811 02:56:28.649827 5122 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 02:56:28.650880 5122 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 02:56:28.659894 5130 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:56:28.660004 5122 server_base.cc:1034] running on GCE node
W20260811 02:56:28.659894 5132 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:56:28.659897 5129 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:56:28.660754 5122 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 02:56:28.662158 5122 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 02:56:28.663381 5122 hybrid_clock.cc:648] HybridClock initialized: now 1786416988663356 us; error 48 us; skew 500 ppm
Aug 11 02:56:28 dist-test-slave-9gcw krb5kdc[5108](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416988, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 02:56:28.672492 5122 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260811 02:56:28.673671 5122 webserver.cc:460] Webserver started at http://127.0.0.1:45619/ using document root <none> and password file <none>
I20260811 02:56:28.674300 5122 fs_manager.cc:359] Metadata directory not provided
I20260811 02:56:28.674412 5122 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 02:56:28.674695 5122 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 02:56:28.677073 5122 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786416707705494-12843-0/minicluster-data/master-0/data/instance:
uuid: "66c83717ff314c038ff7b5c08c398b11"
format_stamp: "Formatted at 2026-08-11 02:56:28 on dist-test-slave-9gcw"
server_key: "df8d3862242a3f255fd9ac143e8c941a"
server_key_iv: "9ccda90cc7d062e1a9a39491baae7d8d"
server_key_version: "encryptionkey@0"
I20260811 02:56:28.677789 5122 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786416707705494-12843-0/minicluster-data/master-0/wal/instance:
uuid: "66c83717ff314c038ff7b5c08c398b11"
format_stamp: "Formatted at 2026-08-11 02:56:28 on dist-test-slave-9gcw"
server_key: "df8d3862242a3f255fd9ac143e8c941a"
server_key_iv: "9ccda90cc7d062e1a9a39491baae7d8d"
server_key_version: "encryptionkey@0"
I20260811 02:56:28.681972 5122 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260811 02:56:28.684551 5139 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 02:56:28.685927 5122 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 02:56:28.686074 5122 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786416707705494-12843-0/minicluster-data/master-0/data,/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786416707705494-12843-0/minicluster-data/master-0/wal
uuid: "66c83717ff314c038ff7b5c08c398b11"
format_stamp: "Formatted at 2026-08-11 02:56:28 on dist-test-slave-9gcw"
server_key: "df8d3862242a3f255fd9ac143e8c941a"
server_key_iv: "9ccda90cc7d062e1a9a39491baae7d8d"
server_key_version: "encryptionkey@0"
I20260811 02:56:28.686195 5122 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786416707705494-12843-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786416707705494-12843-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786416707705494-12843-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 02:56:28.699088 5122 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 02:56:28.702740 5122 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 02:56:28.703007 5122 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 02:56:28.710939 5122 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:45165
I20260811 02:56:28.710968 5196 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:45165 every 8 connection(s)
I20260811 02:56:28.711956 5122 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786416707705494-12843-0/minicluster-data/master-0/data/info.pb
I20260811 02:56:28.714983 5197 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 02:56:28.720098 12843 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDawEwS/build/debug/bin/kudu as pid 5122
I20260811 02:56:28.720247 12843 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786416707705494-12843-0/minicluster-data/master-0/wal/instance
I20260811 02:56:28.720585 12843 external_mini_cluster.cc:1428] Setting key f5a712480e00150f75f3863e14a6be30
I20260811 02:56:28.720849 5197 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 66c83717ff314c038ff7b5c08c398b11: Bootstrap starting.
I20260811 02:56:28.723490 5197 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 66c83717ff314c038ff7b5c08c398b11: Neither blocks nor log segments found. Creating new log.
I20260811 02:56:28.724447 5197 log.cc:824] T 00000000000000000000000000000000 P 66c83717ff314c038ff7b5c08c398b11: Log is configured to *not* fsync() on all Append() calls
I20260811 02:56:28.726771 5197 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 66c83717ff314c038ff7b5c08c398b11: No bootstrap required, opened a new log
Aug 11 02:56:28 dist-test-slave-9gcw krb5kdc[5108](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416988, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260811 02:56:28.731326 5197 raft_consensus.cc:348] T 00000000000000000000000000000000 P 66c83717ff314c038ff7b5c08c398b11 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "66c83717ff314c038ff7b5c08c398b11" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45165 } }
I20260811 02:56:28.731588 5197 raft_consensus.cc:374] T 00000000000000000000000000000000 P 66c83717ff314c038ff7b5c08c398b11 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 02:56:28.731652 5197 raft_consensus.cc:729] T 00000000000000000000000000000000 P 66c83717ff314c038ff7b5c08c398b11 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 66c83717ff314c038ff7b5c08c398b11, State: Initialized, Role: FOLLOWER
I20260811 02:56:28.732205 5197 consensus_queue.cc:260] T 00000000000000000000000000000000 P 66c83717ff314c038ff7b5c08c398b11 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "66c83717ff314c038ff7b5c08c398b11" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45165 } }
I20260811 02:56:28.732409 5197 raft_consensus.cc:388] T 00000000000000000000000000000000 P 66c83717ff314c038ff7b5c08c398b11 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 02:56:28.732496 5197 raft_consensus.cc:482] T 00000000000000000000000000000000 P 66c83717ff314c038ff7b5c08c398b11 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 02:56:28.732609 5197 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 66c83717ff314c038ff7b5c08c398b11 [term 0 FOLLOWER]: Advancing to term 1
I20260811 02:56:28.733846 5197 raft_consensus.cc:504] T 00000000000000000000000000000000 P 66c83717ff314c038ff7b5c08c398b11 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "66c83717ff314c038ff7b5c08c398b11" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45165 } }
I20260811 02:56:28.734238 5197 leader_election.cc:302] T 00000000000000000000000000000000 P 66c83717ff314c038ff7b5c08c398b11 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 66c83717ff314c038ff7b5c08c398b11; no voters:
I20260811 02:56:28.734619 5197 leader_election.cc:288] T 00000000000000000000000000000000 P 66c83717ff314c038ff7b5c08c398b11 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 02:56:28.734798 5203 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 66c83717ff314c038ff7b5c08c398b11 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 02:56:28.735069 5203 raft_consensus.cc:686] T 00000000000000000000000000000000 P 66c83717ff314c038ff7b5c08c398b11 [term 1 LEADER]: Becoming Leader. State: Replica: 66c83717ff314c038ff7b5c08c398b11, State: Running, Role: LEADER
I20260811 02:56:28.735404 5203 consensus_queue.cc:237] T 00000000000000000000000000000000 P 66c83717ff314c038ff7b5c08c398b11 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "66c83717ff314c038ff7b5c08c398b11" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45165 } }
I20260811 02:56:28.736099 5197 sys_catalog.cc:564] T 00000000000000000000000000000000 P 66c83717ff314c038ff7b5c08c398b11 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 02:56:28.737632 5200 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:56:28.722012 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.180:39987 (local address 127.0.0.1:45165)
0811 02:56:28.722456 (+ 444us) server_negotiation.cc:206] Beginning negotiation
0811 02:56:28.722472 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0811 02:56:28.722510 (+ 38us) server_negotiation.cc:407] Connection header received
0811 02:56:28.723226 (+ 716us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:56:28.723245 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:56:28.723659 (+ 414us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:56:28.723960 (+ 301us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:56:28.724984 (+ 1024us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:56:28.726156 (+ 1172us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:56:28.727388 (+ 1232us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:56:28.728011 (+ 623us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:56:28.728138 (+ 127us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:56:28.731274 (+ 3136us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:56:28.731279 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:56:28.731292 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:56:28.731319 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:56:28.733404 (+ 2085us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:56:28.733989 (+ 585us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:56:28.733995 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:56:28.734001 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:56:28.734168 (+ 167us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:56:28.734605 (+ 437us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:56:28.734608 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:56:28.734610 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:56:28.735177 (+ 567us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:56:28.736016 (+ 839us) server_negotiation.cc:1035] Waiting for connection context
0811 02:56:28.736197 (+ 181us) server_negotiation.cc:299] Negotiation successful
0811 02:56:28.736443 (+ 246us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":292,"thread_start_us":151,"threads_started":1}
I20260811 02:56:28.738135 5203 sys_catalog.cc:455] T 00000000000000000000000000000000 P 66c83717ff314c038ff7b5c08c398b11 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 66c83717ff314c038ff7b5c08c398b11. Latest consensus state: current_term: 1 leader_uuid: "66c83717ff314c038ff7b5c08c398b11" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "66c83717ff314c038ff7b5c08c398b11" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45165 } } }
I20260811 02:56:28.738279 5203 sys_catalog.cc:458] T 00000000000000000000000000000000 P 66c83717ff314c038ff7b5c08c398b11 [sys.catalog]: This master's current role is: LEADER
I20260811 02:56:28.738831 5205 sys_catalog.cc:455] T 00000000000000000000000000000000 P 66c83717ff314c038ff7b5c08c398b11 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "66c83717ff314c038ff7b5c08c398b11" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "66c83717ff314c038ff7b5c08c398b11" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45165 } } }
I20260811 02:56:28.738937 5205 sys_catalog.cc:458] T 00000000000000000000000000000000 P 66c83717ff314c038ff7b5c08c398b11 [sys.catalog]: This master's current role is: LEADER
W20260811 02:56:28.741189 5219 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 66c83717ff314c038ff7b5c08c398b11: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 02:56:28.741276 5219 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 02:56:28.741343 5220 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 02:56:28.741386 12843 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 02:56:28.742015 5220 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 02:56:28.747746 5220 catalog_manager.cc:1347] Generated new cluster ID: 7fd62a27dcb84fdc9d4efa067a7f89e3
I20260811 02:56:28.747819 5220 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 02:56:28.752533 5200 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:56:28.743706 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.180:35041 (local address 127.0.0.1:45165)
0811 02:56:28.743820 (+ 114us) server_negotiation.cc:206] Beginning negotiation
0811 02:56:28.743824 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 02:56:28.744086 (+ 262us) server_negotiation.cc:407] Connection header received
0811 02:56:28.744213 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:56:28.744216 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:56:28.744264 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:56:28.744351 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:56:28.745258 (+ 907us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:56:28.745931 (+ 673us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:56:28.747019 (+ 1088us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:56:28.747401 (+ 382us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:56:28.747512 (+ 111us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:56:28.749121 (+ 1609us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:56:28.749125 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:56:28.749128 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:56:28.749180 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:56:28.750716 (+ 1536us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:56:28.751216 (+ 500us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:56:28.751219 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:56:28.751221 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:56:28.751306 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:56:28.751646 (+ 340us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:56:28.751649 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:56:28.751651 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:56:28.751835 (+ 184us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:56:28.751991 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0811 02:56:28.752265 (+ 274us) server_negotiation.cc:299] Negotiation successful
0811 02:56:28.752362 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260811 02:56:28.756438 5220 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 02:56:28.757467 5220 catalog_manager.cc:1504] Loading token signing keys...
I20260811 02:56:28.766670 5220 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 66c83717ff314c038ff7b5c08c398b11: Generated new TSK 0
I20260811 02:56:28.767436 5220 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 02:56:28.776305 12843 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDawEwS/build/debug/bin/kudu with pid 5122
W20260811 02:56:28.787107 5106 connection.cc:535] client connection to 127.0.0.1:45165 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:45165): Connection reset by peer (error 104)
2026-08-11T02:56:28Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/9 (1540 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/10
Loading random data
Initializing database '/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786416707705494-12843-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 02:56:28 dist-test-slave-9gcw krb5kdc[5233](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 02:56:28 dist-test-slave-9gcw krb5kdc[5233](info): set up 2 sockets
Aug 11 02:56:28 dist-test-slave-9gcw krb5kdc[5233](info): commencing operation
krb5kdc: starting...
W20260811 02:56:29.965402 12843 mini_kdc.cc:121] Time spent starting KDC: real 1.163s user 0.002s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786416707705494-12843-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786416707705494-12843-0/krb5kdc/test-user.keytab.
Aug 11 02:56:30 dist-test-slave-9gcw krb5kdc[5233](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416990, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T02:56:30Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T02:56:30Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786416707705494-12843-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786416707705494-12843-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260811 02:56:30.108636 12843 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDawEwS/build/debug/bin/kudu
/tmp/dist-test-taskDawEwS/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786416707705494-12843-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786416707705494-12843-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786416707705494-12843-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786416707705494-12843-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43859
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786416707705494-12843-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:33573
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:43859
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786416707705494-12843-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786416707705494-12843-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786416707705494-12843-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 02:56:30.225646 5247 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 02:56:30.225970 5247 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 02:56:30.226048 5247 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 02:56:30.230125 5247 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 02:56:30.230211 5247 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 02:56:30.230247 5247 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 02:56:30.230304 5247 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 02:56:30.230341 5247 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 02:56:30.235579 5247 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:33573
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786416707705494-12843-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786416707705494-12843-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:43859
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786416707705494-12843-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43859
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786416707705494-12843-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5247
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786416707705494-12843-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 02:09:49 UTC on ccycloud.quasar-fsnqqc.root.comops.site
build id 4579570
I20260811 02:56:30.236752 5247 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 02:56:30.237864 5247 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 02:56:30.246654 5257 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:56:30.246673 5253 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:56:30.246716 5254 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:56:30.246945 5247 server_base.cc:1034] running on GCE node
I20260811 02:56:30.247687 5247 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 02:56:30.248754 5247 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 02:56:30.249946 5247 hybrid_clock.cc:648] HybridClock initialized: now 1786416990249916 us; error 41 us; skew 500 ppm
Aug 11 02:56:30 dist-test-slave-9gcw krb5kdc[5233](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416990, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 02:56:30.252960 5247 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260811 02:56:30.253979 5247 webserver.cc:460] Webserver started at http://127.0.0.1:41161/ using document root <none> and password file <none>
I20260811 02:56:30.254621 5247 fs_manager.cc:359] Metadata directory not provided
I20260811 02:56:30.254691 5247 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 02:56:30.254961 5247 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 02:56:30.257438 5247 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786416707705494-12843-0/minicluster-data/master-0/data/instance:
uuid: "1e42b1d3adc44060aa5a637475812d3c"
format_stamp: "Formatted at 2026-08-11 02:56:30 on dist-test-slave-9gcw"
server_key: "3528514f86094e68ecde5ccd18d874fa"
server_key_iv: "4ba0ae316f1bbcabce721b0d1d3014ca"
server_key_version: "encryptionkey@0"
I20260811 02:56:30.258100 5247 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786416707705494-12843-0/minicluster-data/master-0/wal/instance:
uuid: "1e42b1d3adc44060aa5a637475812d3c"
format_stamp: "Formatted at 2026-08-11 02:56:30 on dist-test-slave-9gcw"
server_key: "3528514f86094e68ecde5ccd18d874fa"
server_key_iv: "4ba0ae316f1bbcabce721b0d1d3014ca"
server_key_version: "encryptionkey@0"
I20260811 02:56:30.262401 5247 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260811 02:56:30.265281 5264 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 02:56:30.266862 5247 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 02:56:30.267030 5247 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786416707705494-12843-0/minicluster-data/master-0/data,/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786416707705494-12843-0/minicluster-data/master-0/wal
uuid: "1e42b1d3adc44060aa5a637475812d3c"
format_stamp: "Formatted at 2026-08-11 02:56:30 on dist-test-slave-9gcw"
server_key: "3528514f86094e68ecde5ccd18d874fa"
server_key_iv: "4ba0ae316f1bbcabce721b0d1d3014ca"
server_key_version: "encryptionkey@0"
I20260811 02:56:30.267172 5247 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786416707705494-12843-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786416707705494-12843-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786416707705494-12843-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 02:56:30.303503 5247 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 02:56:30.307579 5247 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 02:56:30.307880 5247 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 02:56:30.315743 5247 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:43859
I20260811 02:56:30.315798 5323 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:43859 every 8 connection(s)
I20260811 02:56:30.316802 5247 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786416707705494-12843-0/minicluster-data/master-0/data/info.pb
I20260811 02:56:30.317063 12843 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDawEwS/build/debug/bin/kudu as pid 5247
I20260811 02:56:30.317257 12843 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786416707705494-12843-0/minicluster-data/master-0/wal/instance
I20260811 02:56:30.317658 12843 external_mini_cluster.cc:1428] Setting key 1f027b65ac236442c6f476e732f25ed0
I20260811 02:56:30.320379 5324 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 02:56:30.326380 5324 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1e42b1d3adc44060aa5a637475812d3c: Bootstrap starting.
I20260811 02:56:30.327472 5327 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:56:30.319150 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.180:32821 (local address 127.0.0.1:43859)
0811 02:56:30.319717 (+ 567us) server_negotiation.cc:206] Beginning negotiation
0811 02:56:30.319740 (+ 23us) server_negotiation.cc:399] Waiting for connection header
0811 02:56:30.319955 (+ 215us) server_negotiation.cc:407] Connection header received
0811 02:56:30.320816 (+ 861us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:56:30.320834 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:56:30.321250 (+ 416us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:56:30.321595 (+ 345us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:56:30.322020 (+ 425us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:56:30.323028 (+ 1008us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:56:30.323966 (+ 938us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:56:30.324455 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:56:30.324632 (+ 177us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:56:30.325666 (+ 1034us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:56:30.325671 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:56:30.325776 (+ 105us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 02:56:30.326176 (+ 400us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:56:30.326242 (+ 66us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 02:56:30.326267 (+ 25us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:56:30.326415 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0811 02:56:30.326600 (+ 185us) server_negotiation.cc:299] Negotiation successful
0811 02:56:30.326754 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":333,"thread_start_us":155,"threads_started":1}
I20260811 02:56:30.329047 5324 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1e42b1d3adc44060aa5a637475812d3c: Neither blocks nor log segments found. Creating new log.
I20260811 02:56:30.330008 5324 log.cc:824] T 00000000000000000000000000000000 P 1e42b1d3adc44060aa5a637475812d3c: Log is configured to *not* fsync() on all Append() calls
I20260811 02:56:30.330765 12843 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 02:56:30.332165 5324 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1e42b1d3adc44060aa5a637475812d3c: No bootstrap required, opened a new log
I20260811 02:56:30.335888 5324 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1e42b1d3adc44060aa5a637475812d3c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1e42b1d3adc44060aa5a637475812d3c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43859 } }
I20260811 02:56:30.336146 5324 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1e42b1d3adc44060aa5a637475812d3c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 02:56:30.336210 5324 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1e42b1d3adc44060aa5a637475812d3c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1e42b1d3adc44060aa5a637475812d3c, State: Initialized, Role: FOLLOWER
I20260811 02:56:30.336707 5324 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1e42b1d3adc44060aa5a637475812d3c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1e42b1d3adc44060aa5a637475812d3c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43859 } }
I20260811 02:56:30.336884 5324 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1e42b1d3adc44060aa5a637475812d3c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 02:56:30.336980 5324 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1e42b1d3adc44060aa5a637475812d3c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 02:56:30.337091 5324 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1e42b1d3adc44060aa5a637475812d3c [term 0 FOLLOWER]: Advancing to term 1
I20260811 02:56:30.337328 5327 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:56:30.332899 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.180:33867 (local address 127.0.0.1:43859)
0811 02:56:30.333023 (+ 124us) server_negotiation.cc:206] Beginning negotiation
0811 02:56:30.333027 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 02:56:30.333305 (+ 278us) server_negotiation.cc:407] Connection header received
0811 02:56:30.333448 (+ 143us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:56:30.333452 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:56:30.333497 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:56:30.333595 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:56:30.333889 (+ 294us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:56:30.334757 (+ 868us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:56:30.335633 (+ 876us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:56:30.335877 (+ 244us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:56:30.335990 (+ 113us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:56:30.336838 (+ 848us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:56:30.336842 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:56:30.336845 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 02:56:30.336857 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:56:30.336884 (+ 27us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 02:56:30.336897 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:56:30.336979 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0811 02:56:30.337107 (+ 128us) server_negotiation.cc:299] Negotiation successful
0811 02:56:30.337160 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260811 02:56:30.338299 5324 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1e42b1d3adc44060aa5a637475812d3c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1e42b1d3adc44060aa5a637475812d3c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43859 } }
I20260811 02:56:30.338681 5324 leader_election.cc:302] T 00000000000000000000000000000000 P 1e42b1d3adc44060aa5a637475812d3c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1e42b1d3adc44060aa5a637475812d3c; no voters:
I20260811 02:56:30.339004 5324 leader_election.cc:288] T 00000000000000000000000000000000 P 1e42b1d3adc44060aa5a637475812d3c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 02:56:30.339156 5339 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1e42b1d3adc44060aa5a637475812d3c [term 1 FOLLOWER]: Leader election won for term 1
I20260811 02:56:30.339426 5339 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1e42b1d3adc44060aa5a637475812d3c [term 1 LEADER]: Becoming Leader. State: Replica: 1e42b1d3adc44060aa5a637475812d3c, State: Running, Role: LEADER
I20260811 02:56:30.339846 5339 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1e42b1d3adc44060aa5a637475812d3c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1e42b1d3adc44060aa5a637475812d3c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43859 } }
I20260811 02:56:30.340266 5324 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1e42b1d3adc44060aa5a637475812d3c [sys.catalog]: configured and running, proceeding with master startup.
I20260811 02:56:30.341817 5341 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1e42b1d3adc44060aa5a637475812d3c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1e42b1d3adc44060aa5a637475812d3c. Latest consensus state: current_term: 1 leader_uuid: "1e42b1d3adc44060aa5a637475812d3c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1e42b1d3adc44060aa5a637475812d3c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43859 } } }
I20260811 02:56:30.341836 5340 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1e42b1d3adc44060aa5a637475812d3c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1e42b1d3adc44060aa5a637475812d3c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1e42b1d3adc44060aa5a637475812d3c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43859 } } }
I20260811 02:56:30.341975 5340 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1e42b1d3adc44060aa5a637475812d3c [sys.catalog]: This master's current role is: LEADER
I20260811 02:56:30.341975 5341 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1e42b1d3adc44060aa5a637475812d3c [sys.catalog]: This master's current role is: LEADER
I20260811 02:56:30.342875 5354 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 02:56:30.346504 5354 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 02:56:30.352448 5354 catalog_manager.cc:1347] Generated new cluster ID: 321f353c96ae4d4d9b633015670cbc4c
I20260811 02:56:30.352528 5354 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 02:56:30.369021 5354 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 02:56:30.370070 5354 catalog_manager.cc:1504] Loading token signing keys...
I20260811 02:56:30.375151 5354 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1e42b1d3adc44060aa5a637475812d3c: Generated new TSK 0
I20260811 02:56:30.375890 5354 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 02:56:30.384907 12843 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDawEwS/build/debug/bin/kudu with pid 5247
W20260811 02:56:30.395119 5231 connection.cc:535] client connection to 127.0.0.1:43859 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:43859): Connection reset by peer (error 104)
2026-08-11T02:56:30Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/10 (1608 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/11
Loading random data
Initializing database '/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786416707705494-12843-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 02:56:30 dist-test-slave-9gcw krb5kdc[5361](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 02:56:30 dist-test-slave-9gcw krb5kdc[5361](info): set up 2 sockets
Aug 11 02:56:30 dist-test-slave-9gcw krb5kdc[5361](info): commencing operation
krb5kdc: starting...
W20260811 02:56:31.625761 12843 mini_kdc.cc:121] Time spent starting KDC: real 1.216s user 0.002s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786416707705494-12843-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786416707705494-12843-0/krb5kdc/test-user.keytab.
Aug 11 02:56:31 dist-test-slave-9gcw krb5kdc[5361](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416991, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T02:56:31Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T02:56:31Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786416707705494-12843-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786416707705494-12843-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260811 02:56:31.773265 12843 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDawEwS/build/debug/bin/kudu
/tmp/dist-test-taskDawEwS/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786416707705494-12843-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786416707705494-12843-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786416707705494-12843-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786416707705494-12843-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46323
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786416707705494-12843-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:39215
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:46323
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786416707705494-12843-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786416707705494-12843-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786416707705494-12843-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 02:56:31.890793 5376 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 02:56:31.891105 5376 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 02:56:31.891196 5376 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 02:56:31.895201 5376 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 02:56:31.895298 5376 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 02:56:31.895334 5376 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 02:56:31.895395 5376 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 02:56:31.895431 5376 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 02:56:31.900962 5376 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:39215
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786416707705494-12843-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786416707705494-12843-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:46323
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786416707705494-12843-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46323
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786416707705494-12843-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5376
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786416707705494-12843-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 02:09:49 UTC on ccycloud.quasar-fsnqqc.root.comops.site
build id 4579570
I20260811 02:56:31.902386 5376 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 02:56:31.903465 5376 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 02:56:31.912783 5385 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:56:31.912873 5387 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:56:31.912797 5383 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:56:31.912987 5376 server_base.cc:1034] running on GCE node
I20260811 02:56:31.913816 5376 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 02:56:31.915093 5376 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 02:56:31.916322 5376 hybrid_clock.cc:648] HybridClock initialized: now 1786416991916289 us; error 58 us; skew 500 ppm
Aug 11 02:56:31 dist-test-slave-9gcw krb5kdc[5361](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416991, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 02:56:31.919440 5376 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260811 02:56:31.920477 5376 webserver.cc:460] Webserver started at http://127.0.0.1:39841/ using document root <none> and password file <none>
I20260811 02:56:31.921092 5376 fs_manager.cc:359] Metadata directory not provided
I20260811 02:56:31.921233 5376 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 02:56:31.921478 5376 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 02:56:31.923776 5376 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786416707705494-12843-0/minicluster-data/master-0/data/instance:
uuid: "c8556c8e646d4d18a0aaf6fc54ea1ed7"
format_stamp: "Formatted at 2026-08-11 02:56:31 on dist-test-slave-9gcw"
server_key: "ac95cff658f64658b0e63208682eb82e"
server_key_iv: "91b1b6c89a3111da5868bb02da63562e"
server_key_version: "encryptionkey@0"
I20260811 02:56:31.924361 5376 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786416707705494-12843-0/minicluster-data/master-0/wal/instance:
uuid: "c8556c8e646d4d18a0aaf6fc54ea1ed7"
format_stamp: "Formatted at 2026-08-11 02:56:31 on dist-test-slave-9gcw"
server_key: "ac95cff658f64658b0e63208682eb82e"
server_key_iv: "91b1b6c89a3111da5868bb02da63562e"
server_key_version: "encryptionkey@0"
I20260811 02:56:31.928386 5376 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260811 02:56:31.931136 5394 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 02:56:31.932750 5376 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260811 02:56:31.932921 5376 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786416707705494-12843-0/minicluster-data/master-0/data,/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786416707705494-12843-0/minicluster-data/master-0/wal
uuid: "c8556c8e646d4d18a0aaf6fc54ea1ed7"
format_stamp: "Formatted at 2026-08-11 02:56:31 on dist-test-slave-9gcw"
server_key: "ac95cff658f64658b0e63208682eb82e"
server_key_iv: "91b1b6c89a3111da5868bb02da63562e"
server_key_version: "encryptionkey@0"
I20260811 02:56:31.933060 5376 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786416707705494-12843-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786416707705494-12843-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786416707705494-12843-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 02:56:31.938207 5376 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 02:56:31.941737 5376 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 02:56:31.941949 5376 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 02:56:31.949501 5376 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:46323
I20260811 02:56:31.949525 5456 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:46323 every 8 connection(s)
I20260811 02:56:31.950520 5376 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786416707705494-12843-0/minicluster-data/master-0/data/info.pb
I20260811 02:56:31.951601 12843 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDawEwS/build/debug/bin/kudu as pid 5376
I20260811 02:56:31.951745 12843 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786416707705494-12843-0/minicluster-data/master-0/wal/instance
I20260811 02:56:31.952070 12843 external_mini_cluster.cc:1428] Setting key 86bfe5dc72dc6c729acc182242049204
I20260811 02:56:31.954172 5457 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 02:56:31.957729 5460 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:56:31.953601 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.180:40131 (local address 127.0.0.1:46323)
0811 02:56:31.954086 (+ 485us) server_negotiation.cc:206] Beginning negotiation
0811 02:56:31.954102 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0811 02:56:31.954137 (+ 35us) server_negotiation.cc:407] Connection header received
0811 02:56:31.954909 (+ 772us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:56:31.954928 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:56:31.955229 (+ 301us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:56:31.955588 (+ 359us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:56:31.956467 (+ 879us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:56:31.956471 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:56:31.956482 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 02:56:31.956493 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:56:31.956547 (+ 54us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 02:56:31.956567 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:56:31.956714 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0811 02:56:31.956816 (+ 102us) server_negotiation.cc:299] Negotiation successful
0811 02:56:31.956981 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":342,"thread_start_us":165,"threads_started":1}
I20260811 02:56:31.960907 12843 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 02:56:31.961675 5457 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c8556c8e646d4d18a0aaf6fc54ea1ed7: Bootstrap starting.
I20260811 02:56:31.964651 5457 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c8556c8e646d4d18a0aaf6fc54ea1ed7: Neither blocks nor log segments found. Creating new log.
I20260811 02:56:31.965133 5460 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:56:31.963118 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.180:51965 (local address 127.0.0.1:46323)
0811 02:56:31.963237 (+ 119us) server_negotiation.cc:206] Beginning negotiation
0811 02:56:31.963242 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 02:56:31.963433 (+ 191us) server_negotiation.cc:407] Connection header received
0811 02:56:31.963580 (+ 147us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:56:31.963583 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:56:31.963618 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:56:31.963782 (+ 164us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:56:31.964577 (+ 795us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:56:31.964580 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:56:31.964582 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 02:56:31.964594 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:56:31.964620 (+ 26us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 02:56:31.964631 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:56:31.964839 (+ 208us) server_negotiation.cc:1035] Waiting for connection context
0811 02:56:31.964929 (+ 90us) server_negotiation.cc:299] Negotiation successful
0811 02:56:31.964985 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260811 02:56:31.965754 5457 log.cc:824] T 00000000000000000000000000000000 P c8556c8e646d4d18a0aaf6fc54ea1ed7: Log is configured to *not* fsync() on all Append() calls
I20260811 02:56:31.967926 5457 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c8556c8e646d4d18a0aaf6fc54ea1ed7: No bootstrap required, opened a new log
I20260811 02:56:31.971972 5457 raft_consensus.cc:348] T 00000000000000000000000000000000 P c8556c8e646d4d18a0aaf6fc54ea1ed7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c8556c8e646d4d18a0aaf6fc54ea1ed7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46323 } }
I20260811 02:56:31.972209 5457 raft_consensus.cc:374] T 00000000000000000000000000000000 P c8556c8e646d4d18a0aaf6fc54ea1ed7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 02:56:31.972254 5457 raft_consensus.cc:729] T 00000000000000000000000000000000 P c8556c8e646d4d18a0aaf6fc54ea1ed7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c8556c8e646d4d18a0aaf6fc54ea1ed7, State: Initialized, Role: FOLLOWER
I20260811 02:56:31.972638 5457 consensus_queue.cc:260] T 00000000000000000000000000000000 P c8556c8e646d4d18a0aaf6fc54ea1ed7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c8556c8e646d4d18a0aaf6fc54ea1ed7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46323 } }
I20260811 02:56:31.972755 5457 raft_consensus.cc:388] T 00000000000000000000000000000000 P c8556c8e646d4d18a0aaf6fc54ea1ed7 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 02:56:31.972795 5457 raft_consensus.cc:482] T 00000000000000000000000000000000 P c8556c8e646d4d18a0aaf6fc54ea1ed7 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 02:56:31.972858 5457 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c8556c8e646d4d18a0aaf6fc54ea1ed7 [term 0 FOLLOWER]: Advancing to term 1
I20260811 02:56:31.973855 5457 raft_consensus.cc:504] T 00000000000000000000000000000000 P c8556c8e646d4d18a0aaf6fc54ea1ed7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c8556c8e646d4d18a0aaf6fc54ea1ed7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46323 } }
I20260811 02:56:31.974159 5457 leader_election.cc:302] T 00000000000000000000000000000000 P c8556c8e646d4d18a0aaf6fc54ea1ed7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c8556c8e646d4d18a0aaf6fc54ea1ed7; no voters:
I20260811 02:56:31.974424 5457 leader_election.cc:288] T 00000000000000000000000000000000 P c8556c8e646d4d18a0aaf6fc54ea1ed7 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 02:56:31.974571 5469 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c8556c8e646d4d18a0aaf6fc54ea1ed7 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 02:56:31.974964 5469 raft_consensus.cc:686] T 00000000000000000000000000000000 P c8556c8e646d4d18a0aaf6fc54ea1ed7 [term 1 LEADER]: Becoming Leader. State: Replica: c8556c8e646d4d18a0aaf6fc54ea1ed7, State: Running, Role: LEADER
I20260811 02:56:31.975327 5469 consensus_queue.cc:237] T 00000000000000000000000000000000 P c8556c8e646d4d18a0aaf6fc54ea1ed7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c8556c8e646d4d18a0aaf6fc54ea1ed7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46323 } }
I20260811 02:56:31.975771 5457 sys_catalog.cc:564] T 00000000000000000000000000000000 P c8556c8e646d4d18a0aaf6fc54ea1ed7 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 02:56:31.977075 5470 sys_catalog.cc:455] T 00000000000000000000000000000000 P c8556c8e646d4d18a0aaf6fc54ea1ed7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c8556c8e646d4d18a0aaf6fc54ea1ed7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c8556c8e646d4d18a0aaf6fc54ea1ed7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46323 } } }
I20260811 02:56:31.977260 5470 sys_catalog.cc:458] T 00000000000000000000000000000000 P c8556c8e646d4d18a0aaf6fc54ea1ed7 [sys.catalog]: This master's current role is: LEADER
I20260811 02:56:31.977636 5472 sys_catalog.cc:455] T 00000000000000000000000000000000 P c8556c8e646d4d18a0aaf6fc54ea1ed7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c8556c8e646d4d18a0aaf6fc54ea1ed7. Latest consensus state: current_term: 1 leader_uuid: "c8556c8e646d4d18a0aaf6fc54ea1ed7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c8556c8e646d4d18a0aaf6fc54ea1ed7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46323 } } }
I20260811 02:56:31.977735 5472 sys_catalog.cc:458] T 00000000000000000000000000000000 P c8556c8e646d4d18a0aaf6fc54ea1ed7 [sys.catalog]: This master's current role is: LEADER
W20260811 02:56:31.980518 5491 catalog_manager.cc:1558] T 00000000000000000000000000000000 P c8556c8e646d4d18a0aaf6fc54ea1ed7: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 02:56:31.980588 5491 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 02:56:31.980679 5486 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 02:56:31.981469 5486 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 02:56:31.987470 5486 catalog_manager.cc:1347] Generated new cluster ID: ddc1d5bb89ee412ba0d41cd1b4447b8b
I20260811 02:56:31.987535 5486 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 02:56:32.001286 5486 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 02:56:32.001765 5486 catalog_manager.cc:1504] Loading token signing keys...
I20260811 02:56:32.008041 5486 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c8556c8e646d4d18a0aaf6fc54ea1ed7: Generated new TSK 0
I20260811 02:56:32.008965 5486 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 02:56:32.014209 12843 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDawEwS/build/debug/bin/kudu with pid 5376
2026-08-11T02:56:32Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/11 (1628 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/12
Loading random data
Initializing database '/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786416707705494-12843-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 02:56:32 dist-test-slave-9gcw krb5kdc[5497](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 02:56:32 dist-test-slave-9gcw krb5kdc[5497](info): set up 2 sockets
Aug 11 02:56:32 dist-test-slave-9gcw krb5kdc[5497](info): commencing operation
krb5kdc: starting...
W20260811 02:56:33.178468 12843 mini_kdc.cc:121] Time spent starting KDC: real 1.140s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786416707705494-12843-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786416707705494-12843-0/krb5kdc/test-user.keytab.
Aug 11 02:56:33 dist-test-slave-9gcw krb5kdc[5497](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416993, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T02:56:33Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T02:56:33Z Disabled control of system clock
WARNING: no policy specified for kudu/127.12.138.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.138.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.138.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786416707705494-12843-0/krb5kdc/kudu_127.12.138.254.keytab.
Entry for principal kudu/127.12.138.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786416707705494-12843-0/krb5kdc/kudu_127.12.138.254.keytab.
I20260811 02:56:33.321966 12843 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDawEwS/build/debug/bin/kudu
/tmp/dist-test-taskDawEwS/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786416707705494-12843-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786416707705494-12843-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786416707705494-12843-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786416707705494-12843-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.138.254:39807
--webserver_interface=127.12.138.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786416707705494-12843-0/krb5kdc/kudu_127.12.138.254.keytab
--principal=kudu/127.12.138.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.138.212:39021
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.12.138.254:39807
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786416707705494-12843-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786416707705494-12843-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786416707705494-12843-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 02:56:33.438074 5510 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 02:56:33.438421 5510 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 02:56:33.438508 5510 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 02:56:33.442698 5510 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 02:56:33.442826 5510 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 02:56:33.442867 5510 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 02:56:33.442906 5510 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 02:56:33.442938 5510 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 02:56:33.448535 5510 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.138.212:39021
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786416707705494-12843-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786416707705494-12843-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.12.138.254:39807
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786416707705494-12843-0/krb5kdc/kudu_127.12.138.254.keytab
--principal=kudu/127.12.138.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.138.254:39807
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786416707705494-12843-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.12.138.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5510
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786416707705494-12843-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 02:09:49 UTC on ccycloud.quasar-fsnqqc.root.comops.site
build id 4579570
I20260811 02:56:33.449798 5510 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 02:56:33.450929 5510 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 02:56:33.460186 5516 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:56:33.460274 5517 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:56:33.460331 5510 server_base.cc:1034] running on GCE node
W20260811 02:56:33.460274 5521 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:56:33.461100 5510 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 02:56:33.462363 5510 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 02:56:33.463546 5510 hybrid_clock.cc:648] HybridClock initialized: now 1786416993463494 us; error 65 us; skew 500 ppm
Aug 11 02:56:33 dist-test-slave-9gcw krb5kdc[5497](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416993, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.138.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 02:56:33.466816 5510 init.cc:377] Logged in from keytab as kudu/127.12.138.254@KRBTEST.COM (short username kudu)
I20260811 02:56:33.467768 5510 webserver.cc:460] Webserver started at http://127.12.138.254:46603/ using document root <none> and password file <none>
I20260811 02:56:33.468358 5510 fs_manager.cc:359] Metadata directory not provided
I20260811 02:56:33.468423 5510 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 02:56:33.468652 5510 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 02:56:33.470884 5510 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786416707705494-12843-0/minicluster-data/master-0/data/instance:
uuid: "b41223fcc2dd4dd4b5fc5de4188db42a"
format_stamp: "Formatted at 2026-08-11 02:56:33 on dist-test-slave-9gcw"
server_key: "3d98d6aed747631252e9b65d4bead673"
server_key_iv: "361439704a662924f600e80d7cc6347a"
server_key_version: "encryptionkey@0"
I20260811 02:56:33.471530 5510 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786416707705494-12843-0/minicluster-data/master-0/wal/instance:
uuid: "b41223fcc2dd4dd4b5fc5de4188db42a"
format_stamp: "Formatted at 2026-08-11 02:56:33 on dist-test-slave-9gcw"
server_key: "3d98d6aed747631252e9b65d4bead673"
server_key_iv: "361439704a662924f600e80d7cc6347a"
server_key_version: "encryptionkey@0"
I20260811 02:56:33.475602 5510 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260811 02:56:33.478428 5528 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 02:56:33.479815 5510 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 02:56:33.479967 5510 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786416707705494-12843-0/minicluster-data/master-0/data,/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786416707705494-12843-0/minicluster-data/master-0/wal
uuid: "b41223fcc2dd4dd4b5fc5de4188db42a"
format_stamp: "Formatted at 2026-08-11 02:56:33 on dist-test-slave-9gcw"
server_key: "3d98d6aed747631252e9b65d4bead673"
server_key_iv: "361439704a662924f600e80d7cc6347a"
server_key_version: "encryptionkey@0"
I20260811 02:56:33.480083 5510 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786416707705494-12843-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786416707705494-12843-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786416707705494-12843-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 02:56:33.511787 5510 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 02:56:33.515969 5510 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 02:56:33.516260 5510 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 02:56:33.524559 5510 rpc_server.cc:307] RPC server started. Bound to: 127.12.138.254:39807
I20260811 02:56:33.524585 5587 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.138.254:39807 every 8 connection(s)
I20260811 02:56:33.525734 5510 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786416707705494-12843-0/minicluster-data/master-0/data/info.pb
I20260811 02:56:33.528851 5588 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 02:56:33.530511 12843 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDawEwS/build/debug/bin/kudu as pid 5510
I20260811 02:56:33.530750 12843 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786416707705494-12843-0/minicluster-data/master-0/wal/instance
I20260811 02:56:33.531070 12843 external_mini_cluster.cc:1428] Setting key 17b2fc84fd6d493878c39c7761c0fc59
I20260811 02:56:33.535595 5588 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b41223fcc2dd4dd4b5fc5de4188db42a: Bootstrap starting.
I20260811 02:56:33.538048 5588 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b41223fcc2dd4dd4b5fc5de4188db42a: Neither blocks nor log segments found. Creating new log.
I20260811 02:56:33.538901 5588 log.cc:824] T 00000000000000000000000000000000 P b41223fcc2dd4dd4b5fc5de4188db42a: Log is configured to *not* fsync() on all Append() calls
Aug 11 02:56:33 dist-test-slave-9gcw krb5kdc[5497](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416993, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.12.138.254@KRBTEST.COM
I20260811 02:56:33.541191 5588 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b41223fcc2dd4dd4b5fc5de4188db42a: No bootstrap required, opened a new log
I20260811 02:56:33.545199 5588 raft_consensus.cc:348] T 00000000000000000000000000000000 P b41223fcc2dd4dd4b5fc5de4188db42a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b41223fcc2dd4dd4b5fc5de4188db42a" member_type: VOTER last_known_addr { host: "127.12.138.254" port: 39807 } }
I20260811 02:56:33.545465 5588 raft_consensus.cc:374] T 00000000000000000000000000000000 P b41223fcc2dd4dd4b5fc5de4188db42a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 02:56:33.545553 5588 raft_consensus.cc:729] T 00000000000000000000000000000000 P b41223fcc2dd4dd4b5fc5de4188db42a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b41223fcc2dd4dd4b5fc5de4188db42a, State: Initialized, Role: FOLLOWER
I20260811 02:56:33.545853 5591 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:56:33.532506 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42909 (local address 127.12.138.254:39807)
0811 02:56:33.532982 (+ 476us) server_negotiation.cc:206] Beginning negotiation
0811 02:56:33.532998 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0811 02:56:33.533041 (+ 43us) server_negotiation.cc:407] Connection header received
0811 02:56:33.533770 (+ 729us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:56:33.533787 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:56:33.534119 (+ 332us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:56:33.534459 (+ 340us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:56:33.535317 (+ 858us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:56:33.536365 (+ 1048us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:56:33.537292 (+ 927us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:56:33.537775 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:56:33.537963 (+ 188us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:56:33.540602 (+ 2639us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:56:33.540607 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:56:33.540620 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:56:33.540653 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:56:33.542537 (+ 1884us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:56:33.543118 (+ 581us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:56:33.543124 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:56:33.543130 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:56:33.543252 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:56:33.543681 (+ 429us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:56:33.543684 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:56:33.543685 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:56:33.544093 (+ 408us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:56:33.544340 (+ 247us) server_negotiation.cc:1035] Waiting for connection context
0811 02:56:33.544659 (+ 319us) server_negotiation.cc:299] Negotiation successful
0811 02:56:33.544877 (+ 218us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":333,"thread_start_us":141,"threads_started":1}
I20260811 02:56:33.546123 5588 consensus_queue.cc:260] T 00000000000000000000000000000000 P b41223fcc2dd4dd4b5fc5de4188db42a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b41223fcc2dd4dd4b5fc5de4188db42a" member_type: VOTER last_known_addr { host: "127.12.138.254" port: 39807 } }
I20260811 02:56:33.546278 5588 raft_consensus.cc:388] T 00000000000000000000000000000000 P b41223fcc2dd4dd4b5fc5de4188db42a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 02:56:33.546372 5588 raft_consensus.cc:482] T 00000000000000000000000000000000 P b41223fcc2dd4dd4b5fc5de4188db42a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 02:56:33.546482 5588 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b41223fcc2dd4dd4b5fc5de4188db42a [term 0 FOLLOWER]: Advancing to term 1
I20260811 02:56:33.547775 5588 raft_consensus.cc:504] T 00000000000000000000000000000000 P b41223fcc2dd4dd4b5fc5de4188db42a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b41223fcc2dd4dd4b5fc5de4188db42a" member_type: VOTER last_known_addr { host: "127.12.138.254" port: 39807 } }
I20260811 02:56:33.548172 5588 leader_election.cc:302] T 00000000000000000000000000000000 P b41223fcc2dd4dd4b5fc5de4188db42a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b41223fcc2dd4dd4b5fc5de4188db42a; no voters:
I20260811 02:56:33.548588 5588 leader_election.cc:288] T 00000000000000000000000000000000 P b41223fcc2dd4dd4b5fc5de4188db42a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 02:56:33.548708 5596 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b41223fcc2dd4dd4b5fc5de4188db42a [term 1 FOLLOWER]: Leader election won for term 1
I20260811 02:56:33.548986 5596 raft_consensus.cc:686] T 00000000000000000000000000000000 P b41223fcc2dd4dd4b5fc5de4188db42a [term 1 LEADER]: Becoming Leader. State: Replica: b41223fcc2dd4dd4b5fc5de4188db42a, State: Running, Role: LEADER
I20260811 02:56:33.549420 12843 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 02:56:33.549383 5596 consensus_queue.cc:237] T 00000000000000000000000000000000 P b41223fcc2dd4dd4b5fc5de4188db42a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b41223fcc2dd4dd4b5fc5de4188db42a" member_type: VOTER last_known_addr { host: "127.12.138.254" port: 39807 } }
I20260811 02:56:33.550060 5588 sys_catalog.cc:564] T 00000000000000000000000000000000 P b41223fcc2dd4dd4b5fc5de4188db42a [sys.catalog]: configured and running, proceeding with master startup.
I20260811 02:56:33.552044 5598 sys_catalog.cc:455] T 00000000000000000000000000000000 P b41223fcc2dd4dd4b5fc5de4188db42a [sys.catalog]: SysCatalogTable state changed. Reason: New leader b41223fcc2dd4dd4b5fc5de4188db42a. Latest consensus state: current_term: 1 leader_uuid: "b41223fcc2dd4dd4b5fc5de4188db42a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b41223fcc2dd4dd4b5fc5de4188db42a" member_type: VOTER last_known_addr { host: "127.12.138.254" port: 39807 } } }
I20260811 02:56:33.552191 5598 sys_catalog.cc:458] T 00000000000000000000000000000000 P b41223fcc2dd4dd4b5fc5de4188db42a [sys.catalog]: This master's current role is: LEADER
I20260811 02:56:33.552516 5597 sys_catalog.cc:455] T 00000000000000000000000000000000 P b41223fcc2dd4dd4b5fc5de4188db42a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b41223fcc2dd4dd4b5fc5de4188db42a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b41223fcc2dd4dd4b5fc5de4188db42a" member_type: VOTER last_known_addr { host: "127.12.138.254" port: 39807 } } }
I20260811 02:56:33.553121 5597 sys_catalog.cc:458] T 00000000000000000000000000000000 P b41223fcc2dd4dd4b5fc5de4188db42a [sys.catalog]: This master's current role is: LEADER
W20260811 02:56:33.555505 5618 catalog_manager.cc:1558] T 00000000000000000000000000000000 P b41223fcc2dd4dd4b5fc5de4188db42a: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 02:56:33.555622 5618 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 02:56:33.555745 5617 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 02:56:33.556617 5617 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 02:56:33.562264 5591 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:56:33.552063 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44855 (local address 127.12.138.254:39807)
0811 02:56:33.552194 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0811 02:56:33.552199 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 02:56:33.552376 (+ 177us) server_negotiation.cc:407] Connection header received
0811 02:56:33.552541 (+ 165us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:56:33.552545 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:56:33.552594 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:56:33.552660 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:56:33.553735 (+ 1075us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:56:33.554500 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:56:33.555700 (+ 1200us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:56:33.556027 (+ 327us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:56:33.556182 (+ 155us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:56:33.557994 (+ 1812us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:56:33.557998 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:56:33.558001 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:56:33.558036 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:56:33.559923 (+ 1887us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:56:33.560585 (+ 662us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:56:33.560589 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:56:33.560591 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:56:33.560700 (+ 109us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:56:33.561212 (+ 512us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:56:33.561216 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:56:33.561218 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:56:33.561464 (+ 246us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:56:33.561630 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0811 02:56:33.561994 (+ 364us) server_negotiation.cc:299] Negotiation successful
0811 02:56:33.562106 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260811 02:56:33.563401 5617 catalog_manager.cc:1347] Generated new cluster ID: 254fcb808d024e6b9be3ef338586d85b
I20260811 02:56:33.563460 5617 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 02:56:33.584088 5617 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 02:56:33.585507 5617 catalog_manager.cc:1504] Loading token signing keys...
I20260811 02:56:33.596735 5617 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b41223fcc2dd4dd4b5fc5de4188db42a: Generated new TSK 0
I20260811 02:56:33.597728 5617 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 02:56:33.608642 12843 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDawEwS/build/debug/bin/kudu with pid 5510
W20260811 02:56:33.617911 5495 connection.cc:535] client connection to 127.12.138.254:39807 recv error: Network error: failed to read from TLS socket (remote: 127.12.138.254:39807): Connection reset by peer (error 104)
2026-08-11T02:56:33Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/12 (1593 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/13
Loading random data
Initializing database '/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786416707705494-12843-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 02:56:33 dist-test-slave-9gcw krb5kdc[5624](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 02:56:33 dist-test-slave-9gcw krb5kdc[5624](info): set up 2 sockets
Aug 11 02:56:33 dist-test-slave-9gcw krb5kdc[5624](info): commencing operation
krb5kdc: starting...
W20260811 02:56:34.765825 12843 mini_kdc.cc:121] Time spent starting KDC: real 1.133s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786416707705494-12843-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786416707705494-12843-0/krb5kdc/test-user.keytab.
Aug 11 02:56:34 dist-test-slave-9gcw krb5kdc[5624](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416994, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T02:56:34Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T02:56:34Z Disabled control of system clock
WARNING: no policy specified for kudu/127.12.138.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.138.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.138.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786416707705494-12843-0/krb5kdc/kudu_127.12.138.254.keytab.
Entry for principal kudu/127.12.138.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786416707705494-12843-0/krb5kdc/kudu_127.12.138.254.keytab.
I20260811 02:56:34.906545 12843 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDawEwS/build/debug/bin/kudu
/tmp/dist-test-taskDawEwS/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786416707705494-12843-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786416707705494-12843-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786416707705494-12843-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786416707705494-12843-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.138.254:38981
--webserver_interface=127.12.138.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786416707705494-12843-0/krb5kdc/kudu_127.12.138.254.keytab
--principal=kudu/127.12.138.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.138.212:37467
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.12.138.254:38981
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786416707705494-12843-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786416707705494-12843-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786416707705494-12843-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 02:56:35.032719 5639 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 02:56:35.033291 5639 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 02:56:35.033413 5639 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 02:56:35.038080 5639 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 02:56:35.038152 5639 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 02:56:35.038177 5639 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 02:56:35.038196 5639 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 02:56:35.038214 5639 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 02:56:35.044471 5639 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.138.212:37467
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786416707705494-12843-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786416707705494-12843-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.12.138.254:38981
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786416707705494-12843-0/krb5kdc/kudu_127.12.138.254.keytab
--principal=kudu/127.12.138.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.138.254:38981
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786416707705494-12843-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.12.138.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5639
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786416707705494-12843-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 02:09:49 UTC on ccycloud.quasar-fsnqqc.root.comops.site
build id 4579570
I20260811 02:56:35.045764 5639 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 02:56:35.046835 5639 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 02:56:35.056130 5645 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:56:35.056124 5644 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:56:35.056166 5647 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:56:35.056505 5639 server_base.cc:1034] running on GCE node
I20260811 02:56:35.057233 5639 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 02:56:35.058481 5639 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 02:56:35.059648 5639 hybrid_clock.cc:648] HybridClock initialized: now 1786416995059621 us; error 51 us; skew 500 ppm
Aug 11 02:56:35 dist-test-slave-9gcw krb5kdc[5624](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416995, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.138.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 02:56:35.067790 5639 init.cc:377] Logged in from keytab as kudu/127.12.138.254@KRBTEST.COM (short username kudu)
I20260811 02:56:35.068934 5639 webserver.cc:460] Webserver started at http://127.12.138.254:37707/ using document root <none> and password file <none>
I20260811 02:56:35.069602 5639 fs_manager.cc:359] Metadata directory not provided
I20260811 02:56:35.069674 5639 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 02:56:35.069905 5639 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 02:56:35.072145 5639 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786416707705494-12843-0/minicluster-data/master-0/data/instance:
uuid: "24d6488bb1bd471f995e1d0c85fb666b"
format_stamp: "Formatted at 2026-08-11 02:56:35 on dist-test-slave-9gcw"
server_key: "51769930de52da8fceb02c2f52870b6a"
server_key_iv: "b76bc14ba175ab52e1b7887d7ac599fd"
server_key_version: "encryptionkey@0"
I20260811 02:56:35.072806 5639 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786416707705494-12843-0/minicluster-data/master-0/wal/instance:
uuid: "24d6488bb1bd471f995e1d0c85fb666b"
format_stamp: "Formatted at 2026-08-11 02:56:35 on dist-test-slave-9gcw"
server_key: "51769930de52da8fceb02c2f52870b6a"
server_key_iv: "b76bc14ba175ab52e1b7887d7ac599fd"
server_key_version: "encryptionkey@0"
I20260811 02:56:35.077558 5639 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.008s sys 0.000s
I20260811 02:56:35.080204 5654 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 02:56:35.081634 5639 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 02:56:35.081771 5639 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786416707705494-12843-0/minicluster-data/master-0/data,/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786416707705494-12843-0/minicluster-data/master-0/wal
uuid: "24d6488bb1bd471f995e1d0c85fb666b"
format_stamp: "Formatted at 2026-08-11 02:56:35 on dist-test-slave-9gcw"
server_key: "51769930de52da8fceb02c2f52870b6a"
server_key_iv: "b76bc14ba175ab52e1b7887d7ac599fd"
server_key_version: "encryptionkey@0"
I20260811 02:56:35.081890 5639 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786416707705494-12843-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786416707705494-12843-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786416707705494-12843-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 02:56:35.100358 5639 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 02:56:35.103801 5639 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 02:56:35.104030 5639 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 02:56:35.111935 5639 rpc_server.cc:307] RPC server started. Bound to: 127.12.138.254:38981
I20260811 02:56:35.111959 5714 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.138.254:38981 every 8 connection(s)
I20260811 02:56:35.112938 5639 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786416707705494-12843-0/minicluster-data/master-0/data/info.pb
I20260811 02:56:35.114729 12843 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDawEwS/build/debug/bin/kudu as pid 5639
I20260811 02:56:35.114884 12843 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786416707705494-12843-0/minicluster-data/master-0/wal/instance
I20260811 02:56:35.115217 12843 external_mini_cluster.cc:1428] Setting key 7b5cb31af478f0a5e49a060578ad2140
I20260811 02:56:35.116217 5715 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 02:56:35.122853 5715 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 24d6488bb1bd471f995e1d0c85fb666b: Bootstrap starting.
I20260811 02:56:35.125025 5718 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:56:35.116708 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34609 (local address 127.12.138.254:38981)
0811 02:56:35.117197 (+ 489us) server_negotiation.cc:206] Beginning negotiation
0811 02:56:35.117217 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0811 02:56:35.117264 (+ 47us) server_negotiation.cc:407] Connection header received
0811 02:56:35.118082 (+ 818us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:56:35.118102 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:56:35.118466 (+ 364us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:56:35.118755 (+ 289us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:56:35.119147 (+ 392us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:56:35.120242 (+ 1095us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:56:35.121549 (+ 1307us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:56:35.122071 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:56:35.122208 (+ 137us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:56:35.123228 (+ 1020us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:56:35.123233 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:56:35.123245 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 02:56:35.123636 (+ 391us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:56:35.123707 (+ 71us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 02:56:35.123740 (+ 33us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:56:35.123852 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0811 02:56:35.124096 (+ 244us) server_negotiation.cc:299] Negotiation successful
0811 02:56:35.124281 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":303,"thread_start_us":164,"threads_started":1}
I20260811 02:56:35.125379 5715 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 24d6488bb1bd471f995e1d0c85fb666b: Neither blocks nor log segments found. Creating new log.
I20260811 02:56:35.126165 5715 log.cc:824] T 00000000000000000000000000000000 P 24d6488bb1bd471f995e1d0c85fb666b: Log is configured to *not* fsync() on all Append() calls
I20260811 02:56:35.128348 5715 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 24d6488bb1bd471f995e1d0c85fb666b: No bootstrap required, opened a new log
I20260811 02:56:35.128368 12843 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 02:56:35.132862 5715 raft_consensus.cc:348] T 00000000000000000000000000000000 P 24d6488bb1bd471f995e1d0c85fb666b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "24d6488bb1bd471f995e1d0c85fb666b" member_type: VOTER last_known_addr { host: "127.12.138.254" port: 38981 } }
I20260811 02:56:35.133211 5715 raft_consensus.cc:374] T 00000000000000000000000000000000 P 24d6488bb1bd471f995e1d0c85fb666b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 02:56:35.133287 5715 raft_consensus.cc:729] T 00000000000000000000000000000000 P 24d6488bb1bd471f995e1d0c85fb666b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 24d6488bb1bd471f995e1d0c85fb666b, State: Initialized, Role: FOLLOWER
I20260811 02:56:35.133811 5715 consensus_queue.cc:260] T 00000000000000000000000000000000 P 24d6488bb1bd471f995e1d0c85fb666b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "24d6488bb1bd471f995e1d0c85fb666b" member_type: VOTER last_known_addr { host: "127.12.138.254" port: 38981 } }
I20260811 02:56:35.133941 5715 raft_consensus.cc:388] T 00000000000000000000000000000000 P 24d6488bb1bd471f995e1d0c85fb666b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 02:56:35.134028 5715 raft_consensus.cc:482] T 00000000000000000000000000000000 P 24d6488bb1bd471f995e1d0c85fb666b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 02:56:35.134169 5715 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 24d6488bb1bd471f995e1d0c85fb666b [term 0 FOLLOWER]: Advancing to term 1
I20260811 02:56:35.135053 5718 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:56:35.130617 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40973 (local address 127.12.138.254:38981)
0811 02:56:35.130770 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0811 02:56:35.130775 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 02:56:35.130900 (+ 125us) server_negotiation.cc:407] Connection header received
0811 02:56:35.131000 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:56:35.131003 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:56:35.131046 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:56:35.131124 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:56:35.131355 (+ 231us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:56:35.132179 (+ 824us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:56:35.133052 (+ 873us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:56:35.133418 (+ 366us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:56:35.133554 (+ 136us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:56:35.134593 (+ 1039us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:56:35.134596 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:56:35.134599 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 02:56:35.134612 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:56:35.134650 (+ 38us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 02:56:35.134665 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:56:35.134757 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0811 02:56:35.134887 (+ 130us) server_negotiation.cc:299] Negotiation successful
0811 02:56:35.134924 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260811 02:56:35.135443 5715 raft_consensus.cc:504] T 00000000000000000000000000000000 P 24d6488bb1bd471f995e1d0c85fb666b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "24d6488bb1bd471f995e1d0c85fb666b" member_type: VOTER last_known_addr { host: "127.12.138.254" port: 38981 } }
I20260811 02:56:35.135813 5715 leader_election.cc:302] T 00000000000000000000000000000000 P 24d6488bb1bd471f995e1d0c85fb666b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 24d6488bb1bd471f995e1d0c85fb666b; no voters:
I20260811 02:56:35.136160 5715 leader_election.cc:288] T 00000000000000000000000000000000 P 24d6488bb1bd471f995e1d0c85fb666b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 02:56:35.136222 5725 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 24d6488bb1bd471f995e1d0c85fb666b [term 1 FOLLOWER]: Leader election won for term 1
I20260811 02:56:35.136439 5725 raft_consensus.cc:686] T 00000000000000000000000000000000 P 24d6488bb1bd471f995e1d0c85fb666b [term 1 LEADER]: Becoming Leader. State: Replica: 24d6488bb1bd471f995e1d0c85fb666b, State: Running, Role: LEADER
I20260811 02:56:35.136750 5725 consensus_queue.cc:237] T 00000000000000000000000000000000 P 24d6488bb1bd471f995e1d0c85fb666b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "24d6488bb1bd471f995e1d0c85fb666b" member_type: VOTER last_known_addr { host: "127.12.138.254" port: 38981 } }
I20260811 02:56:35.137552 5715 sys_catalog.cc:564] T 00000000000000000000000000000000 P 24d6488bb1bd471f995e1d0c85fb666b [sys.catalog]: configured and running, proceeding with master startup.
I20260811 02:56:35.138564 5726 sys_catalog.cc:455] T 00000000000000000000000000000000 P 24d6488bb1bd471f995e1d0c85fb666b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 24d6488bb1bd471f995e1d0c85fb666b. Latest consensus state: current_term: 1 leader_uuid: "24d6488bb1bd471f995e1d0c85fb666b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "24d6488bb1bd471f995e1d0c85fb666b" member_type: VOTER last_known_addr { host: "127.12.138.254" port: 38981 } } }
I20260811 02:56:35.138696 5726 sys_catalog.cc:458] T 00000000000000000000000000000000 P 24d6488bb1bd471f995e1d0c85fb666b [sys.catalog]: This master's current role is: LEADER
I20260811 02:56:35.139046 5727 sys_catalog.cc:455] T 00000000000000000000000000000000 P 24d6488bb1bd471f995e1d0c85fb666b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "24d6488bb1bd471f995e1d0c85fb666b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "24d6488bb1bd471f995e1d0c85fb666b" member_type: VOTER last_known_addr { host: "127.12.138.254" port: 38981 } } }
I20260811 02:56:35.139137 5737 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 02:56:35.139138 5727 sys_catalog.cc:458] T 00000000000000000000000000000000 P 24d6488bb1bd471f995e1d0c85fb666b [sys.catalog]: This master's current role is: LEADER
I20260811 02:56:35.142853 5737 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 02:56:35.148684 5737 catalog_manager.cc:1347] Generated new cluster ID: 4afe579b36154e76925c05dbe7475d46
I20260811 02:56:35.148762 5737 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 02:56:35.171834 5737 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 02:56:35.173265 5737 catalog_manager.cc:1504] Loading token signing keys...
I20260811 02:56:35.188488 5737 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 24d6488bb1bd471f995e1d0c85fb666b: Generated new TSK 0
I20260811 02:56:35.189448 5737 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 02:56:35.200654 12843 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDawEwS/build/debug/bin/kudu with pid 5639
W20260811 02:56:35.208127 5622 connection.cc:535] client connection to 127.12.138.254:38981 recv error: Network error: failed to read from TLS socket (remote: 127.12.138.254:38981): Connection reset by peer (error 104)
2026-08-11T02:56:35Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/13 (1590 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/14
Loading random data
Initializing database '/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786416707705494-12843-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 02:56:35 dist-test-slave-9gcw krb5kdc[5750](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 02:56:35 dist-test-slave-9gcw krb5kdc[5750](info): set up 2 sockets
Aug 11 02:56:35 dist-test-slave-9gcw krb5kdc[5750](info): commencing operation
krb5kdc: starting...
W20260811 02:56:36.352322 12843 mini_kdc.cc:121] Time spent starting KDC: real 1.130s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786416707705494-12843-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786416707705494-12843-0/krb5kdc/test-user.keytab.
Aug 11 02:56:36 dist-test-slave-9gcw krb5kdc[5750](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416996, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T02:56:36Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T02:56:36Z Disabled control of system clock
WARNING: no policy specified for kudu/127.12.138.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.138.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.138.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786416707705494-12843-0/krb5kdc/kudu_127.12.138.254.keytab.
Entry for principal kudu/127.12.138.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786416707705494-12843-0/krb5kdc/kudu_127.12.138.254.keytab.
I20260811 02:56:36.497781 12843 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDawEwS/build/debug/bin/kudu
/tmp/dist-test-taskDawEwS/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786416707705494-12843-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786416707705494-12843-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786416707705494-12843-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786416707705494-12843-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.138.254:41069
--webserver_interface=127.12.138.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786416707705494-12843-0/krb5kdc/kudu_127.12.138.254.keytab
--principal=kudu/127.12.138.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.138.212:43333
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.12.138.254:41069
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786416707705494-12843-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786416707705494-12843-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786416707705494-12843-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 02:56:36.623713 5764 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 02:56:36.624063 5764 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 02:56:36.624154 5764 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 02:56:36.628288 5764 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 02:56:36.628391 5764 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 02:56:36.628427 5764 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 02:56:36.628464 5764 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 02:56:36.628501 5764 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 02:56:36.634114 5764 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.138.212:43333
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786416707705494-12843-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786416707705494-12843-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.12.138.254:41069
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786416707705494-12843-0/krb5kdc/kudu_127.12.138.254.keytab
--principal=kudu/127.12.138.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.138.254:41069
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786416707705494-12843-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.12.138.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5764
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786416707705494-12843-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 02:09:49 UTC on ccycloud.quasar-fsnqqc.root.comops.site
build id 4579570
I20260811 02:56:36.635257 5764 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 02:56:36.636370 5764 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 02:56:36.645270 5774 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:56:36.645304 5772 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:56:36.645264 5771 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:56:36.645437 5764 server_base.cc:1034] running on GCE node
I20260811 02:56:36.646322 5764 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 02:56:36.647401 5764 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 02:56:36.648597 5764 hybrid_clock.cc:648] HybridClock initialized: now 1786416996648582 us; error 33 us; skew 500 ppm
Aug 11 02:56:36 dist-test-slave-9gcw krb5kdc[5750](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416996, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.138.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 02:56:36.651692 5764 init.cc:377] Logged in from keytab as kudu/127.12.138.254@KRBTEST.COM (short username kudu)
I20260811 02:56:36.652614 5764 webserver.cc:460] Webserver started at http://127.12.138.254:42097/ using document root <none> and password file <none>
I20260811 02:56:36.653275 5764 fs_manager.cc:359] Metadata directory not provided
I20260811 02:56:36.653374 5764 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 02:56:36.653617 5764 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 02:56:36.655691 5764 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786416707705494-12843-0/minicluster-data/master-0/data/instance:
uuid: "0b86c33f734a45e18b8ee6059c7e8278"
format_stamp: "Formatted at 2026-08-11 02:56:36 on dist-test-slave-9gcw"
server_key: "9152ad85895539072bb69de6fcac5fb8"
server_key_iv: "3af98a8a256d6d825b943ec3355215be"
server_key_version: "encryptionkey@0"
I20260811 02:56:36.656275 5764 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786416707705494-12843-0/minicluster-data/master-0/wal/instance:
uuid: "0b86c33f734a45e18b8ee6059c7e8278"
format_stamp: "Formatted at 2026-08-11 02:56:36 on dist-test-slave-9gcw"
server_key: "9152ad85895539072bb69de6fcac5fb8"
server_key_iv: "3af98a8a256d6d825b943ec3355215be"
server_key_version: "encryptionkey@0"
I20260811 02:56:36.660073 5764 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260811 02:56:36.662866 5783 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 02:56:36.664355 5764 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260811 02:56:36.664528 5764 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786416707705494-12843-0/minicluster-data/master-0/data,/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786416707705494-12843-0/minicluster-data/master-0/wal
uuid: "0b86c33f734a45e18b8ee6059c7e8278"
format_stamp: "Formatted at 2026-08-11 02:56:36 on dist-test-slave-9gcw"
server_key: "9152ad85895539072bb69de6fcac5fb8"
server_key_iv: "3af98a8a256d6d825b943ec3355215be"
server_key_version: "encryptionkey@0"
I20260811 02:56:36.664686 5764 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786416707705494-12843-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786416707705494-12843-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786416707705494-12843-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 02:56:36.669939 5764 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 02:56:36.673902 5764 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 02:56:36.674134 5764 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 02:56:36.682078 5764 rpc_server.cc:307] RPC server started. Bound to: 127.12.138.254:41069
I20260811 02:56:36.682250 5840 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.138.254:41069 every 8 connection(s)
I20260811 02:56:36.683233 5764 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786416707705494-12843-0/minicluster-data/master-0/data/info.pb
I20260811 02:56:36.685973 12843 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDawEwS/build/debug/bin/kudu as pid 5764
I20260811 02:56:36.686151 12843 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786416707705494-12843-0/minicluster-data/master-0/wal/instance
I20260811 02:56:36.686486 12843 external_mini_cluster.cc:1428] Setting key bb7887afa37f132d019cb7ccd6867592
I20260811 02:56:36.686486 5841 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 02:56:36.692634 5844 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:56:36.688157 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51573 (local address 127.12.138.254:41069)
0811 02:56:36.688597 (+ 440us) server_negotiation.cc:206] Beginning negotiation
0811 02:56:36.688624 (+ 27us) server_negotiation.cc:399] Waiting for connection header
0811 02:56:36.688667 (+ 43us) server_negotiation.cc:407] Connection header received
0811 02:56:36.689500 (+ 833us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:56:36.689524 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:56:36.689901 (+ 377us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:56:36.690301 (+ 400us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:56:36.691169 (+ 868us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:56:36.691175 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:56:36.691190 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 02:56:36.691210 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:56:36.691283 (+ 73us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 02:56:36.691315 (+ 32us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:56:36.691459 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0811 02:56:36.691526 (+ 67us) server_negotiation.cc:299] Negotiation successful
0811 02:56:36.691693 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":290,"thread_start_us":133,"threads_started":1}
I20260811 02:56:36.694042 5841 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0b86c33f734a45e18b8ee6059c7e8278: Bootstrap starting.
I20260811 02:56:36.695706 12843 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 02:56:36.696622 5841 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0b86c33f734a45e18b8ee6059c7e8278: Neither blocks nor log segments found. Creating new log.
I20260811 02:56:36.697587 5841 log.cc:824] T 00000000000000000000000000000000 P 0b86c33f734a45e18b8ee6059c7e8278: Log is configured to *not* fsync() on all Append() calls
I20260811 02:56:36.699981 5844 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:56:36.698209 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40601 (local address 127.12.138.254:41069)
0811 02:56:36.698347 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0811 02:56:36.698352 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 02:56:36.698490 (+ 138us) server_negotiation.cc:407] Connection header received
0811 02:56:36.698610 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:56:36.698613 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:56:36.698650 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:56:36.698744 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:56:36.699526 (+ 782us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:56:36.699529 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:56:36.699531 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 02:56:36.699543 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:56:36.699569 (+ 26us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 02:56:36.699580 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:56:36.699674 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0811 02:56:36.699777 (+ 103us) server_negotiation.cc:299] Negotiation successful
0811 02:56:36.699813 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260811 02:56:36.700201 5841 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0b86c33f734a45e18b8ee6059c7e8278: No bootstrap required, opened a new log
I20260811 02:56:36.703751 5841 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0b86c33f734a45e18b8ee6059c7e8278 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0b86c33f734a45e18b8ee6059c7e8278" member_type: VOTER last_known_addr { host: "127.12.138.254" port: 41069 } }
I20260811 02:56:36.704002 5841 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0b86c33f734a45e18b8ee6059c7e8278 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 02:56:36.704063 5841 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0b86c33f734a45e18b8ee6059c7e8278 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0b86c33f734a45e18b8ee6059c7e8278, State: Initialized, Role: FOLLOWER
I20260811 02:56:36.704535 5841 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0b86c33f734a45e18b8ee6059c7e8278 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0b86c33f734a45e18b8ee6059c7e8278" member_type: VOTER last_known_addr { host: "127.12.138.254" port: 41069 } }
I20260811 02:56:36.704704 5841 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0b86c33f734a45e18b8ee6059c7e8278 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 02:56:36.704777 5841 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0b86c33f734a45e18b8ee6059c7e8278 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 02:56:36.704876 5841 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0b86c33f734a45e18b8ee6059c7e8278 [term 0 FOLLOWER]: Advancing to term 1
I20260811 02:56:36.706050 5841 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0b86c33f734a45e18b8ee6059c7e8278 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0b86c33f734a45e18b8ee6059c7e8278" member_type: VOTER last_known_addr { host: "127.12.138.254" port: 41069 } }
I20260811 02:56:36.706405 5841 leader_election.cc:302] T 00000000000000000000000000000000 P 0b86c33f734a45e18b8ee6059c7e8278 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0b86c33f734a45e18b8ee6059c7e8278; no voters:
I20260811 02:56:36.706763 5841 leader_election.cc:288] T 00000000000000000000000000000000 P 0b86c33f734a45e18b8ee6059c7e8278 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 02:56:36.706925 5851 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0b86c33f734a45e18b8ee6059c7e8278 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 02:56:36.707190 5851 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0b86c33f734a45e18b8ee6059c7e8278 [term 1 LEADER]: Becoming Leader. State: Replica: 0b86c33f734a45e18b8ee6059c7e8278, State: Running, Role: LEADER
I20260811 02:56:36.707525 5851 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0b86c33f734a45e18b8ee6059c7e8278 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0b86c33f734a45e18b8ee6059c7e8278" member_type: VOTER last_known_addr { host: "127.12.138.254" port: 41069 } }
I20260811 02:56:36.708037 5841 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0b86c33f734a45e18b8ee6059c7e8278 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 02:56:36.709605 5853 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0b86c33f734a45e18b8ee6059c7e8278 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0b86c33f734a45e18b8ee6059c7e8278" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0b86c33f734a45e18b8ee6059c7e8278" member_type: VOTER last_known_addr { host: "127.12.138.254" port: 41069 } } }
I20260811 02:56:36.709610 5854 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0b86c33f734a45e18b8ee6059c7e8278 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0b86c33f734a45e18b8ee6059c7e8278. Latest consensus state: current_term: 1 leader_uuid: "0b86c33f734a45e18b8ee6059c7e8278" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0b86c33f734a45e18b8ee6059c7e8278" member_type: VOTER last_known_addr { host: "127.12.138.254" port: 41069 } } }
I20260811 02:56:36.709771 5853 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0b86c33f734a45e18b8ee6059c7e8278 [sys.catalog]: This master's current role is: LEADER
I20260811 02:56:36.709771 5854 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0b86c33f734a45e18b8ee6059c7e8278 [sys.catalog]: This master's current role is: LEADER
W20260811 02:56:36.713498 5868 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 0b86c33f734a45e18b8ee6059c7e8278: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 02:56:36.713585 5868 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 02:56:36.713675 5867 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 02:56:36.714630 5867 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 02:56:36.721817 5867 catalog_manager.cc:1347] Generated new cluster ID: c8183896bc1f40c1b376099fcf4a00d3
I20260811 02:56:36.721906 5867 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 02:56:36.734741 5867 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 02:56:36.735110 5867 catalog_manager.cc:1504] Loading token signing keys...
I20260811 02:56:36.742292 5867 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0b86c33f734a45e18b8ee6059c7e8278: Generated new TSK 0
I20260811 02:56:36.743070 5867 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 02:56:36.751922 12843 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDawEwS/build/debug/bin/kudu with pid 5764
2026-08-11T02:56:36Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/14 (1554 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/15
Loading random data
Initializing database '/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786416707705494-12843-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 02:56:36 dist-test-slave-9gcw krb5kdc[5876](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 02:56:36 dist-test-slave-9gcw krb5kdc[5876](info): set up 2 sockets
Aug 11 02:56:36 dist-test-slave-9gcw krb5kdc[5876](info): commencing operation
krb5kdc: starting...
W20260811 02:56:37.924432 12843 mini_kdc.cc:121] Time spent starting KDC: real 1.146s user 0.005s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786416707705494-12843-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786416707705494-12843-0/krb5kdc/test-user.keytab.
Aug 11 02:56:38 dist-test-slave-9gcw krb5kdc[5876](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416998, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T02:56:38Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T02:56:38Z Disabled control of system clock
WARNING: no policy specified for kudu/127.12.138.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.138.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.138.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786416707705494-12843-0/krb5kdc/kudu_127.12.138.254.keytab.
Entry for principal kudu/127.12.138.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786416707705494-12843-0/krb5kdc/kudu_127.12.138.254.keytab.
I20260811 02:56:38.070744 12843 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDawEwS/build/debug/bin/kudu
/tmp/dist-test-taskDawEwS/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786416707705494-12843-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786416707705494-12843-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786416707705494-12843-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786416707705494-12843-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.138.254:44979
--webserver_interface=127.12.138.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786416707705494-12843-0/krb5kdc/kudu_127.12.138.254.keytab
--principal=kudu/127.12.138.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.138.212:35617
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.12.138.254:44979
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786416707705494-12843-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786416707705494-12843-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786416707705494-12843-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 02:56:38.192691 5892 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 02:56:38.193063 5892 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 02:56:38.193207 5892 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 02:56:38.197309 5892 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 02:56:38.197414 5892 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 02:56:38.197453 5892 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 02:56:38.197494 5892 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 02:56:38.197527 5892 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 02:56:38.203722 5892 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.138.212:35617
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786416707705494-12843-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786416707705494-12843-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.12.138.254:44979
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786416707705494-12843-0/krb5kdc/kudu_127.12.138.254.keytab
--principal=kudu/127.12.138.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.138.254:44979
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786416707705494-12843-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.12.138.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5892
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786416707705494-12843-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 02:09:49 UTC on ccycloud.quasar-fsnqqc.root.comops.site
build id 4579570
I20260811 02:56:38.204911 5892 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 02:56:38.206054 5892 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 02:56:38.215225 5898 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:56:38.215308 5892 server_base.cc:1034] running on GCE node
W20260811 02:56:38.215488 5897 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:56:38.215499 5900 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:56:38.216132 5892 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 02:56:38.217427 5892 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 02:56:38.218600 5892 hybrid_clock.cc:648] HybridClock initialized: now 1786416998218579 us; error 33 us; skew 500 ppm
Aug 11 02:56:38 dist-test-slave-9gcw krb5kdc[5876](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416998, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.138.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 02:56:38.221588 5892 init.cc:377] Logged in from keytab as kudu/127.12.138.254@KRBTEST.COM (short username kudu)
I20260811 02:56:38.222532 5892 webserver.cc:460] Webserver started at http://127.12.138.254:40579/ using document root <none> and password file <none>
I20260811 02:56:38.223125 5892 fs_manager.cc:359] Metadata directory not provided
I20260811 02:56:38.223229 5892 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 02:56:38.223474 5892 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 02:56:38.225805 5892 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786416707705494-12843-0/minicluster-data/master-0/data/instance:
uuid: "b400a59c72cf4379ba5247551a7978f0"
format_stamp: "Formatted at 2026-08-11 02:56:38 on dist-test-slave-9gcw"
server_key: "e6e7449796002515203dc603cd457338"
server_key_iv: "f4476424606b0cc91b4dff1bc7606610"
server_key_version: "encryptionkey@0"
I20260811 02:56:38.226420 5892 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786416707705494-12843-0/minicluster-data/master-0/wal/instance:
uuid: "b400a59c72cf4379ba5247551a7978f0"
format_stamp: "Formatted at 2026-08-11 02:56:38 on dist-test-slave-9gcw"
server_key: "e6e7449796002515203dc603cd457338"
server_key_iv: "f4476424606b0cc91b4dff1bc7606610"
server_key_version: "encryptionkey@0"
I20260811 02:56:38.230459 5892 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260811 02:56:38.232990 5909 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 02:56:38.234350 5892 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260811 02:56:38.234521 5892 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786416707705494-12843-0/minicluster-data/master-0/data,/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786416707705494-12843-0/minicluster-data/master-0/wal
uuid: "b400a59c72cf4379ba5247551a7978f0"
format_stamp: "Formatted at 2026-08-11 02:56:38 on dist-test-slave-9gcw"
server_key: "e6e7449796002515203dc603cd457338"
server_key_iv: "f4476424606b0cc91b4dff1bc7606610"
server_key_version: "encryptionkey@0"
I20260811 02:56:38.234644 5892 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786416707705494-12843-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786416707705494-12843-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786416707705494-12843-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 02:56:38.255044 5892 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 02:56:38.258143 5892 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 02:56:38.258368 5892 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 02:56:38.266089 5892 rpc_server.cc:307] RPC server started. Bound to: 127.12.138.254:44979
I20260811 02:56:38.266109 5968 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.138.254:44979 every 8 connection(s)
I20260811 02:56:38.267132 5892 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786416707705494-12843-0/minicluster-data/master-0/data/info.pb
I20260811 02:56:38.269342 12843 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDawEwS/build/debug/bin/kudu as pid 5892
I20260811 02:56:38.269497 12843 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786416707705494-12843-0/minicluster-data/master-0/wal/instance
I20260811 02:56:38.269790 12843 external_mini_cluster.cc:1428] Setting key cccd6ebdbc2a0f3f0a17ec29e76f5912
I20260811 02:56:38.270738 5969 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 02:56:38.278800 5969 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b400a59c72cf4379ba5247551a7978f0: Bootstrap starting.
Aug 11 02:56:38 dist-test-slave-9gcw krb5kdc[5876](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416998, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.12.138.254@KRBTEST.COM
I20260811 02:56:38.281694 5969 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b400a59c72cf4379ba5247551a7978f0: Neither blocks nor log segments found. Creating new log.
I20260811 02:56:38.282682 5969 log.cc:824] T 00000000000000000000000000000000 P b400a59c72cf4379ba5247551a7978f0: Log is configured to *not* fsync() on all Append() calls
I20260811 02:56:38.284951 5969 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b400a59c72cf4379ba5247551a7978f0: No bootstrap required, opened a new log
I20260811 02:56:38.287427 5972 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:56:38.271361 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49829 (local address 127.12.138.254:44979)
0811 02:56:38.271833 (+ 472us) server_negotiation.cc:206] Beginning negotiation
0811 02:56:38.271855 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0811 02:56:38.271899 (+ 44us) server_negotiation.cc:407] Connection header received
0811 02:56:38.272873 (+ 974us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:56:38.272898 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:56:38.273406 (+ 508us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:56:38.273826 (+ 420us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:56:38.274831 (+ 1005us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:56:38.276522 (+ 1691us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:56:38.277703 (+ 1181us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:56:38.278492 (+ 789us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:56:38.278688 (+ 196us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:56:38.281946 (+ 3258us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:56:38.281951 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:56:38.281966 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:56:38.281997 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:56:38.284148 (+ 2151us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:56:38.284700 (+ 552us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:56:38.284712 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:56:38.284719 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:56:38.284856 (+ 137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:56:38.285234 (+ 378us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:56:38.285238 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:56:38.285240 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:56:38.285694 (+ 454us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:56:38.285953 (+ 259us) server_negotiation.cc:1035] Waiting for connection context
0811 02:56:38.286302 (+ 349us) server_negotiation.cc:299] Negotiation successful
0811 02:56:38.286624 (+ 322us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":314,"thread_start_us":140,"threads_started":1}
I20260811 02:56:38.289136 5969 raft_consensus.cc:348] T 00000000000000000000000000000000 P b400a59c72cf4379ba5247551a7978f0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b400a59c72cf4379ba5247551a7978f0" member_type: VOTER last_known_addr { host: "127.12.138.254" port: 44979 } }
I20260811 02:56:38.289547 5969 raft_consensus.cc:374] T 00000000000000000000000000000000 P b400a59c72cf4379ba5247551a7978f0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 02:56:38.289680 5969 raft_consensus.cc:729] T 00000000000000000000000000000000 P b400a59c72cf4379ba5247551a7978f0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b400a59c72cf4379ba5247551a7978f0, State: Initialized, Role: FOLLOWER
I20260811 02:56:38.290222 5969 consensus_queue.cc:260] T 00000000000000000000000000000000 P b400a59c72cf4379ba5247551a7978f0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b400a59c72cf4379ba5247551a7978f0" member_type: VOTER last_known_addr { host: "127.12.138.254" port: 44979 } }
I20260811 02:56:38.290412 5969 raft_consensus.cc:388] T 00000000000000000000000000000000 P b400a59c72cf4379ba5247551a7978f0 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 02:56:38.290509 5969 raft_consensus.cc:482] T 00000000000000000000000000000000 P b400a59c72cf4379ba5247551a7978f0 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 02:56:38.290635 5969 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b400a59c72cf4379ba5247551a7978f0 [term 0 FOLLOWER]: Advancing to term 1
I20260811 02:56:38.290702 12843 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 02:56:38.291736 5969 raft_consensus.cc:504] T 00000000000000000000000000000000 P b400a59c72cf4379ba5247551a7978f0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b400a59c72cf4379ba5247551a7978f0" member_type: VOTER last_known_addr { host: "127.12.138.254" port: 44979 } }
I20260811 02:56:38.292176 5969 leader_election.cc:302] T 00000000000000000000000000000000 P b400a59c72cf4379ba5247551a7978f0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b400a59c72cf4379ba5247551a7978f0; no voters:
I20260811 02:56:38.292541 5969 leader_election.cc:288] T 00000000000000000000000000000000 P b400a59c72cf4379ba5247551a7978f0 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 02:56:38.292685 5978 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b400a59c72cf4379ba5247551a7978f0 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 02:56:38.292971 5978 raft_consensus.cc:686] T 00000000000000000000000000000000 P b400a59c72cf4379ba5247551a7978f0 [term 1 LEADER]: Becoming Leader. State: Replica: b400a59c72cf4379ba5247551a7978f0, State: Running, Role: LEADER
I20260811 02:56:38.293430 5978 consensus_queue.cc:237] T 00000000000000000000000000000000 P b400a59c72cf4379ba5247551a7978f0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b400a59c72cf4379ba5247551a7978f0" member_type: VOTER last_known_addr { host: "127.12.138.254" port: 44979 } }
I20260811 02:56:38.294028 5969 sys_catalog.cc:564] T 00000000000000000000000000000000 P b400a59c72cf4379ba5247551a7978f0 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 02:56:38.295446 5981 sys_catalog.cc:455] T 00000000000000000000000000000000 P b400a59c72cf4379ba5247551a7978f0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b400a59c72cf4379ba5247551a7978f0. Latest consensus state: current_term: 1 leader_uuid: "b400a59c72cf4379ba5247551a7978f0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b400a59c72cf4379ba5247551a7978f0" member_type: VOTER last_known_addr { host: "127.12.138.254" port: 44979 } } }
I20260811 02:56:38.295606 5981 sys_catalog.cc:458] T 00000000000000000000000000000000 P b400a59c72cf4379ba5247551a7978f0 [sys.catalog]: This master's current role is: LEADER
I20260811 02:56:38.296012 5979 sys_catalog.cc:455] T 00000000000000000000000000000000 P b400a59c72cf4379ba5247551a7978f0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b400a59c72cf4379ba5247551a7978f0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b400a59c72cf4379ba5247551a7978f0" member_type: VOTER last_known_addr { host: "127.12.138.254" port: 44979 } } }
I20260811 02:56:38.296125 5979 sys_catalog.cc:458] T 00000000000000000000000000000000 P b400a59c72cf4379ba5247551a7978f0 [sys.catalog]: This master's current role is: LEADER
W20260811 02:56:38.299412 5997 catalog_manager.cc:1558] T 00000000000000000000000000000000 P b400a59c72cf4379ba5247551a7978f0: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 02:56:38.299502 5997 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 02:56:38.299566 5991 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 02:56:38.300524 5991 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 02:56:38.305817 5972 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:56:38.293208 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59803 (local address 127.12.138.254:44979)
0811 02:56:38.293334 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0811 02:56:38.293339 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 02:56:38.293803 (+ 464us) server_negotiation.cc:407] Connection header received
0811 02:56:38.294143 (+ 340us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:56:38.294147 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:56:38.294202 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:56:38.294271 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:56:38.297077 (+ 2806us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:56:38.298048 (+ 971us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:56:38.299417 (+ 1369us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:56:38.299816 (+ 399us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:56:38.299938 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:56:38.301910 (+ 1972us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:56:38.301914 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:56:38.301917 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:56:38.301945 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:56:38.303763 (+ 1818us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:56:38.304267 (+ 504us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:56:38.304271 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:56:38.304274 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:56:38.304370 (+ 96us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:56:38.304804 (+ 434us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:56:38.304809 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:56:38.304811 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:56:38.305030 (+ 219us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:56:38.305250 (+ 220us) server_negotiation.cc:1035] Waiting for connection context
0811 02:56:38.305513 (+ 263us) server_negotiation.cc:299] Negotiation successful
0811 02:56:38.305641 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260811 02:56:38.307590 5991 catalog_manager.cc:1347] Generated new cluster ID: 6413486fa4da442799e18f5b6bd18be0
I20260811 02:56:38.307660 5991 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 02:56:38.314975 5991 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 02:56:38.316322 5991 catalog_manager.cc:1504] Loading token signing keys...
I20260811 02:56:38.334597 5991 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b400a59c72cf4379ba5247551a7978f0: Generated new TSK 0
I20260811 02:56:38.335626 5991 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 02:56:38.340785 12843 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDawEwS/build/debug/bin/kudu with pid 5892
2026-08-11T02:56:38Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/15 (1587 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/16
Loading random data
Initializing database '/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786416707705494-12843-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 02:56:38 dist-test-slave-9gcw krb5kdc[6004](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 02:56:38 dist-test-slave-9gcw krb5kdc[6004](info): set up 2 sockets
Aug 11 02:56:38 dist-test-slave-9gcw krb5kdc[6004](info): commencing operation
krb5kdc: starting...
W20260811 02:56:39.535790 12843 mini_kdc.cc:121] Time spent starting KDC: real 1.170s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786416707705494-12843-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786416707705494-12843-0/krb5kdc/test-user.keytab.
Aug 11 02:56:39 dist-test-slave-9gcw krb5kdc[6004](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416999, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T02:56:39Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T02:56:39Z Disabled control of system clock
WARNING: no policy specified for kudu/127.12.138.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.138.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.138.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786416707705494-12843-0/krb5kdc/kudu_127.12.138.254.keytab.
Entry for principal kudu/127.12.138.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786416707705494-12843-0/krb5kdc/kudu_127.12.138.254.keytab.
I20260811 02:56:39.676462 12843 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDawEwS/build/debug/bin/kudu
/tmp/dist-test-taskDawEwS/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786416707705494-12843-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786416707705494-12843-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786416707705494-12843-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786416707705494-12843-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.138.254:45729
--webserver_interface=127.12.138.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786416707705494-12843-0/krb5kdc/kudu_127.12.138.254.keytab
--principal=kudu/127.12.138.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.138.212:38651
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.12.138.254:45729
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786416707705494-12843-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786416707705494-12843-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786416707705494-12843-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 02:56:39.810493 6017 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 02:56:39.810838 6017 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 02:56:39.810945 6017 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 02:56:39.815169 6017 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 02:56:39.815271 6017 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 02:56:39.815316 6017 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 02:56:39.815395 6017 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 02:56:39.815439 6017 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 02:56:39.820981 6017 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.138.212:38651
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786416707705494-12843-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786416707705494-12843-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.12.138.254:45729
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786416707705494-12843-0/krb5kdc/kudu_127.12.138.254.keytab
--principal=kudu/127.12.138.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.138.254:45729
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786416707705494-12843-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.12.138.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6017
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786416707705494-12843-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 02:09:49 UTC on ccycloud.quasar-fsnqqc.root.comops.site
build id 4579570
I20260811 02:56:39.822203 6017 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 02:56:39.823290 6017 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 02:56:39.832774 6023 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:56:39.832886 6024 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:56:39.832791 6026 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:56:39.833000 6017 server_base.cc:1034] running on GCE node
I20260811 02:56:39.833688 6017 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 02:56:39.835093 6017 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 02:56:39.836323 6017 hybrid_clock.cc:648] HybridClock initialized: now 1786416999836303 us; error 32 us; skew 500 ppm
Aug 11 02:56:39 dist-test-slave-9gcw krb5kdc[6004](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416999, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.138.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 02:56:39.839732 6017 init.cc:377] Logged in from keytab as kudu/127.12.138.254@KRBTEST.COM (short username kudu)
I20260811 02:56:39.840829 6017 webserver.cc:460] Webserver started at http://127.12.138.254:41915/ using document root <none> and password file <none>
I20260811 02:56:39.841558 6017 fs_manager.cc:359] Metadata directory not provided
I20260811 02:56:39.841671 6017 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 02:56:39.841938 6017 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 02:56:39.844269 6017 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786416707705494-12843-0/minicluster-data/master-0/data/instance:
uuid: "9bdf43c280cc4c5f828a901994b03050"
format_stamp: "Formatted at 2026-08-11 02:56:39 on dist-test-slave-9gcw"
server_key: "540660386f3cc2a996e146bedf99af99"
server_key_iv: "493f2afc7dfc4654d58db04380aadbc2"
server_key_version: "encryptionkey@0"
I20260811 02:56:39.844923 6017 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786416707705494-12843-0/minicluster-data/master-0/wal/instance:
uuid: "9bdf43c280cc4c5f828a901994b03050"
format_stamp: "Formatted at 2026-08-11 02:56:39 on dist-test-slave-9gcw"
server_key: "540660386f3cc2a996e146bedf99af99"
server_key_iv: "493f2afc7dfc4654d58db04380aadbc2"
server_key_version: "encryptionkey@0"
I20260811 02:56:39.849370 6017 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260811 02:56:39.852218 6033 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 02:56:39.853820 6017 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 02:56:39.853976 6017 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786416707705494-12843-0/minicluster-data/master-0/data,/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786416707705494-12843-0/minicluster-data/master-0/wal
uuid: "9bdf43c280cc4c5f828a901994b03050"
format_stamp: "Formatted at 2026-08-11 02:56:39 on dist-test-slave-9gcw"
server_key: "540660386f3cc2a996e146bedf99af99"
server_key_iv: "493f2afc7dfc4654d58db04380aadbc2"
server_key_version: "encryptionkey@0"
I20260811 02:56:39.854099 6017 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786416707705494-12843-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786416707705494-12843-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786416707705494-12843-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 02:56:39.886538 6017 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 02:56:39.890388 6017 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 02:56:39.890630 6017 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 02:56:39.898831 6017 rpc_server.cc:307] RPC server started. Bound to: 127.12.138.254:45729
I20260811 02:56:39.898859 6090 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.138.254:45729 every 8 connection(s)
I20260811 02:56:39.899920 6017 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786416707705494-12843-0/minicluster-data/master-0/data/info.pb
I20260811 02:56:39.902889 6091 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 02:56:39.905378 12843 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDawEwS/build/debug/bin/kudu as pid 6017
I20260811 02:56:39.905534 12843 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786416707705494-12843-0/minicluster-data/master-0/wal/instance
I20260811 02:56:39.905848 12843 external_mini_cluster.cc:1428] Setting key 7e2c4a124516e883bccb6c94f5b385b3
I20260811 02:56:39.909389 6091 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9bdf43c280cc4c5f828a901994b03050: Bootstrap starting.
I20260811 02:56:39.911640 6091 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9bdf43c280cc4c5f828a901994b03050: Neither blocks nor log segments found. Creating new log.
I20260811 02:56:39.912411 6091 log.cc:824] T 00000000000000000000000000000000 P 9bdf43c280cc4c5f828a901994b03050: Log is configured to *not* fsync() on all Append() calls
I20260811 02:56:39.914631 6091 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9bdf43c280cc4c5f828a901994b03050: No bootstrap required, opened a new log
I20260811 02:56:39.915210 6094 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:56:39.907355 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50077 (local address 127.12.138.254:45729)
0811 02:56:39.907872 (+ 517us) server_negotiation.cc:206] Beginning negotiation
0811 02:56:39.907887 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0811 02:56:39.907925 (+ 38us) server_negotiation.cc:407] Connection header received
0811 02:56:39.908612 (+ 687us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:56:39.908632 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:56:39.909027 (+ 395us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:56:39.909439 (+ 412us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:56:39.909870 (+ 431us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:56:39.910851 (+ 981us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:56:39.911761 (+ 910us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:56:39.912245 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:56:39.912426 (+ 181us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:56:39.913404 (+ 978us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:56:39.913412 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:56:39.913432 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 02:56:39.913824 (+ 392us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:56:39.913881 (+ 57us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 02:56:39.913905 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:56:39.914025 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0811 02:56:39.914145 (+ 120us) server_negotiation.cc:299] Negotiation successful
0811 02:56:39.914379 (+ 234us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":366,"thread_start_us":146,"threads_started":1}
I20260811 02:56:39.918278 12843 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 02:56:39.919029 6091 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9bdf43c280cc4c5f828a901994b03050 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9bdf43c280cc4c5f828a901994b03050" member_type: VOTER last_known_addr { host: "127.12.138.254" port: 45729 } }
I20260811 02:56:39.919349 6091 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9bdf43c280cc4c5f828a901994b03050 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 02:56:39.919474 6091 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9bdf43c280cc4c5f828a901994b03050 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9bdf43c280cc4c5f828a901994b03050, State: Initialized, Role: FOLLOWER
I20260811 02:56:39.919981 6091 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9bdf43c280cc4c5f828a901994b03050 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9bdf43c280cc4c5f828a901994b03050" member_type: VOTER last_known_addr { host: "127.12.138.254" port: 45729 } }
I20260811 02:56:39.920148 6091 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9bdf43c280cc4c5f828a901994b03050 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 02:56:39.920243 6091 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9bdf43c280cc4c5f828a901994b03050 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 02:56:39.920346 6091 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9bdf43c280cc4c5f828a901994b03050 [term 0 FOLLOWER]: Advancing to term 1
I20260811 02:56:39.921689 6091 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9bdf43c280cc4c5f828a901994b03050 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9bdf43c280cc4c5f828a901994b03050" member_type: VOTER last_known_addr { host: "127.12.138.254" port: 45729 } }
I20260811 02:56:39.922049 6091 leader_election.cc:302] T 00000000000000000000000000000000 P 9bdf43c280cc4c5f828a901994b03050 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9bdf43c280cc4c5f828a901994b03050; no voters:
I20260811 02:56:39.922431 6091 leader_election.cc:288] T 00000000000000000000000000000000 P 9bdf43c280cc4c5f828a901994b03050 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 02:56:39.922541 6104 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9bdf43c280cc4c5f828a901994b03050 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 02:56:39.922766 6104 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9bdf43c280cc4c5f828a901994b03050 [term 1 LEADER]: Becoming Leader. State: Replica: 9bdf43c280cc4c5f828a901994b03050, State: Running, Role: LEADER
I20260811 02:56:39.923087 6104 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9bdf43c280cc4c5f828a901994b03050 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9bdf43c280cc4c5f828a901994b03050" member_type: VOTER last_known_addr { host: "127.12.138.254" port: 45729 } }
I20260811 02:56:39.923918 6091 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9bdf43c280cc4c5f828a901994b03050 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 02:56:39.924430 6104 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9bdf43c280cc4c5f828a901994b03050 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9bdf43c280cc4c5f828a901994b03050. Latest consensus state: current_term: 1 leader_uuid: "9bdf43c280cc4c5f828a901994b03050" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9bdf43c280cc4c5f828a901994b03050" member_type: VOTER last_known_addr { host: "127.12.138.254" port: 45729 } } }
I20260811 02:56:39.924546 6104 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9bdf43c280cc4c5f828a901994b03050 [sys.catalog]: This master's current role is: LEADER
I20260811 02:56:39.924953 6105 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9bdf43c280cc4c5f828a901994b03050 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9bdf43c280cc4c5f828a901994b03050" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9bdf43c280cc4c5f828a901994b03050" member_type: VOTER last_known_addr { host: "127.12.138.254" port: 45729 } } }
I20260811 02:56:39.925079 6105 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9bdf43c280cc4c5f828a901994b03050 [sys.catalog]: This master's current role is: LEADER
I20260811 02:56:39.925380 6109 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 02:56:39.926041 6094 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:56:39.920493 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53675 (local address 127.12.138.254:45729)
0811 02:56:39.920626 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0811 02:56:39.920631 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 02:56:39.920820 (+ 189us) server_negotiation.cc:407] Connection header received
0811 02:56:39.920936 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:56:39.920938 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:56:39.920977 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:56:39.921043 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:56:39.921357 (+ 314us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:56:39.922005 (+ 648us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:56:39.923161 (+ 1156us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:56:39.923485 (+ 324us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:56:39.923564 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:56:39.925456 (+ 1892us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:56:39.925459 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:56:39.925463 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 02:56:39.925475 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:56:39.925506 (+ 31us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 02:56:39.925522 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:56:39.925599 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0811 02:56:39.925809 (+ 210us) server_negotiation.cc:299] Negotiation successful
0811 02:56:39.925883 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260811 02:56:39.928634 6109 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 02:56:39.934116 6109 catalog_manager.cc:1347] Generated new cluster ID: 895bebbfafe34049a5692ba04a486a3e
I20260811 02:56:39.934186 6109 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 02:56:39.966140 6109 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 02:56:39.967222 6109 catalog_manager.cc:1504] Loading token signing keys...
I20260811 02:56:39.976168 6109 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9bdf43c280cc4c5f828a901994b03050: Generated new TSK 0
I20260811 02:56:39.977053 6109 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 02:56:39.990988 12843 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDawEwS/build/debug/bin/kudu with pid 6017
2026-08-11T02:56:39Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/16 (1649 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/17
Loading random data
Initializing database '/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786416707705494-12843-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 02:56:40 dist-test-slave-9gcw krb5kdc[6128](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 02:56:40 dist-test-slave-9gcw krb5kdc[6128](info): set up 2 sockets
Aug 11 02:56:40 dist-test-slave-9gcw krb5kdc[6128](info): commencing operation
krb5kdc: starting...
W20260811 02:56:41.173256 12843 mini_kdc.cc:121] Time spent starting KDC: real 1.158s user 0.000s sys 0.009s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786416707705494-12843-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786416707705494-12843-0/krb5kdc/test-user.keytab.
Aug 11 02:56:41 dist-test-slave-9gcw krb5kdc[6128](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417001, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T02:56:41Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T02:56:41Z Disabled control of system clock
WARNING: no policy specified for kudu/127.12.138.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.138.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.138.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786416707705494-12843-0/krb5kdc/kudu_127.12.138.254.keytab.
Entry for principal kudu/127.12.138.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786416707705494-12843-0/krb5kdc/kudu_127.12.138.254.keytab.
I20260811 02:56:41.317010 12843 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDawEwS/build/debug/bin/kudu
/tmp/dist-test-taskDawEwS/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786416707705494-12843-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786416707705494-12843-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786416707705494-12843-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786416707705494-12843-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.138.254:39201
--webserver_interface=127.12.138.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786416707705494-12843-0/krb5kdc/kudu_127.12.138.254.keytab
--principal=kudu/127.12.138.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.138.212:38267
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.12.138.254:39201
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786416707705494-12843-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786416707705494-12843-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786416707705494-12843-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 02:56:41.435406 6142 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 02:56:41.435705 6142 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 02:56:41.435750 6142 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 02:56:41.439939 6142 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 02:56:41.440011 6142 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 02:56:41.440032 6142 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 02:56:41.440049 6142 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 02:56:41.440064 6142 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 02:56:41.445631 6142 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.138.212:38267
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786416707705494-12843-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786416707705494-12843-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.12.138.254:39201
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786416707705494-12843-0/krb5kdc/kudu_127.12.138.254.keytab
--principal=kudu/127.12.138.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.138.254:39201
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786416707705494-12843-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.12.138.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6142
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786416707705494-12843-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 02:09:49 UTC on ccycloud.quasar-fsnqqc.root.comops.site
build id 4579570
I20260811 02:56:41.447263 6142 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 02:56:41.448487 6142 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 02:56:41.457794 6147 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:56:41.457767 6150 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:56:41.457782 6148 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:56:41.458029 6142 server_base.cc:1034] running on GCE node
I20260811 02:56:41.458719 6142 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 02:56:41.459868 6142 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 02:56:41.461050 6142 hybrid_clock.cc:648] HybridClock initialized: now 1786417001461014 us; error 44 us; skew 500 ppm
Aug 11 02:56:41 dist-test-slave-9gcw krb5kdc[6128](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417001, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.138.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 02:56:41.469477 6142 init.cc:377] Logged in from keytab as kudu/127.12.138.254@KRBTEST.COM (short username kudu)
I20260811 02:56:41.470567 6142 webserver.cc:460] Webserver started at http://127.12.138.254:33495/ using document root <none> and password file <none>
I20260811 02:56:41.471223 6142 fs_manager.cc:359] Metadata directory not provided
I20260811 02:56:41.471325 6142 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 02:56:41.471587 6142 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 02:56:41.473855 6142 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786416707705494-12843-0/minicluster-data/master-0/data/instance:
uuid: "46ae53b938f84fccacdde76c2b52de33"
format_stamp: "Formatted at 2026-08-11 02:56:41 on dist-test-slave-9gcw"
server_key: "45cc03817511b6bac1c9783e3347d432"
server_key_iv: "151d5587cf31f2e5da3c6eb6da635b6a"
server_key_version: "encryptionkey@0"
I20260811 02:56:41.474457 6142 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786416707705494-12843-0/minicluster-data/master-0/wal/instance:
uuid: "46ae53b938f84fccacdde76c2b52de33"
format_stamp: "Formatted at 2026-08-11 02:56:41 on dist-test-slave-9gcw"
server_key: "45cc03817511b6bac1c9783e3347d432"
server_key_iv: "151d5587cf31f2e5da3c6eb6da635b6a"
server_key_version: "encryptionkey@0"
I20260811 02:56:41.478488 6142 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260811 02:56:41.481207 6158 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 02:56:41.482538 6142 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 02:56:41.482692 6142 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786416707705494-12843-0/minicluster-data/master-0/data,/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786416707705494-12843-0/minicluster-data/master-0/wal
uuid: "46ae53b938f84fccacdde76c2b52de33"
format_stamp: "Formatted at 2026-08-11 02:56:41 on dist-test-slave-9gcw"
server_key: "45cc03817511b6bac1c9783e3347d432"
server_key_iv: "151d5587cf31f2e5da3c6eb6da635b6a"
server_key_version: "encryptionkey@0"
I20260811 02:56:41.482815 6142 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786416707705494-12843-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786416707705494-12843-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786416707705494-12843-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 02:56:41.487931 6142 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 02:56:41.491506 6142 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 02:56:41.491720 6142 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 02:56:41.499439 6142 rpc_server.cc:307] RPC server started. Bound to: 127.12.138.254:39201
I20260811 02:56:41.499464 6220 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.138.254:39201 every 8 connection(s)
I20260811 02:56:41.500453 6142 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786416707705494-12843-0/minicluster-data/master-0/data/info.pb
I20260811 02:56:41.503386 6221 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 02:56:41.505388 12843 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDawEwS/build/debug/bin/kudu as pid 6142
I20260811 02:56:41.505540 12843 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786416707705494-12843-0/minicluster-data/master-0/wal/instance
I20260811 02:56:41.505844 12843 external_mini_cluster.cc:1428] Setting key 6fe629ab5f3b9c90ebe35214196dfe18
I20260811 02:56:41.510320 6221 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 46ae53b938f84fccacdde76c2b52de33: Bootstrap starting.
I20260811 02:56:41.511310 6226 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:56:41.507349 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39433 (local address 127.12.138.254:39201)
0811 02:56:41.507804 (+ 455us) server_negotiation.cc:206] Beginning negotiation
0811 02:56:41.507819 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0811 02:56:41.507861 (+ 42us) server_negotiation.cc:407] Connection header received
0811 02:56:41.508524 (+ 663us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:56:41.508542 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:56:41.508828 (+ 286us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:56:41.509197 (+ 369us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:56:41.510059 (+ 862us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:56:41.510064 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:56:41.510075 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 02:56:41.510087 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:56:41.510144 (+ 57us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 02:56:41.510165 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:56:41.510335 (+ 170us) server_negotiation.cc:1035] Waiting for connection context
0811 02:56:41.510409 (+ 74us) server_negotiation.cc:299] Negotiation successful
0811 02:56:41.510577 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":308,"thread_start_us":142,"threads_started":1}
I20260811 02:56:41.512851 6221 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 46ae53b938f84fccacdde76c2b52de33: Neither blocks nor log segments found. Creating new log.
I20260811 02:56:41.513809 6221 log.cc:824] T 00000000000000000000000000000000 P 46ae53b938f84fccacdde76c2b52de33: Log is configured to *not* fsync() on all Append() calls
I20260811 02:56:41.514374 12843 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 02:56:41.516336 6221 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 46ae53b938f84fccacdde76c2b52de33: No bootstrap required, opened a new log
I20260811 02:56:41.518708 6226 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:56:41.516788 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43601 (local address 127.12.138.254:39201)
0811 02:56:41.516936 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0811 02:56:41.516941 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 02:56:41.517214 (+ 273us) server_negotiation.cc:407] Connection header received
0811 02:56:41.517368 (+ 154us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:56:41.517370 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:56:41.517408 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:56:41.517475 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:56:41.518226 (+ 751us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:56:41.518229 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:56:41.518231 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 02:56:41.518241 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:56:41.518267 (+ 26us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 02:56:41.518277 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:56:41.518376 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0811 02:56:41.518461 (+ 85us) server_negotiation.cc:299] Negotiation successful
0811 02:56:41.518530 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260811 02:56:41.520744 6221 raft_consensus.cc:348] T 00000000000000000000000000000000 P 46ae53b938f84fccacdde76c2b52de33 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "46ae53b938f84fccacdde76c2b52de33" member_type: VOTER last_known_addr { host: "127.12.138.254" port: 39201 } }
I20260811 02:56:41.520995 6221 raft_consensus.cc:374] T 00000000000000000000000000000000 P 46ae53b938f84fccacdde76c2b52de33 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 02:56:41.521057 6221 raft_consensus.cc:729] T 00000000000000000000000000000000 P 46ae53b938f84fccacdde76c2b52de33 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 46ae53b938f84fccacdde76c2b52de33, State: Initialized, Role: FOLLOWER
I20260811 02:56:41.521566 6221 consensus_queue.cc:260] T 00000000000000000000000000000000 P 46ae53b938f84fccacdde76c2b52de33 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "46ae53b938f84fccacdde76c2b52de33" member_type: VOTER last_known_addr { host: "127.12.138.254" port: 39201 } }
I20260811 02:56:41.521718 6221 raft_consensus.cc:388] T 00000000000000000000000000000000 P 46ae53b938f84fccacdde76c2b52de33 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 02:56:41.521770 6221 raft_consensus.cc:482] T 00000000000000000000000000000000 P 46ae53b938f84fccacdde76c2b52de33 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 02:56:41.521842 6221 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 46ae53b938f84fccacdde76c2b52de33 [term 0 FOLLOWER]: Advancing to term 1
I20260811 02:56:41.522881 6221 raft_consensus.cc:504] T 00000000000000000000000000000000 P 46ae53b938f84fccacdde76c2b52de33 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "46ae53b938f84fccacdde76c2b52de33" member_type: VOTER last_known_addr { host: "127.12.138.254" port: 39201 } }
I20260811 02:56:41.523193 6221 leader_election.cc:302] T 00000000000000000000000000000000 P 46ae53b938f84fccacdde76c2b52de33 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 46ae53b938f84fccacdde76c2b52de33; no voters:
I20260811 02:56:41.523448 6221 leader_election.cc:288] T 00000000000000000000000000000000 P 46ae53b938f84fccacdde76c2b52de33 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 02:56:41.523567 6233 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 46ae53b938f84fccacdde76c2b52de33 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 02:56:41.523866 6233 raft_consensus.cc:686] T 00000000000000000000000000000000 P 46ae53b938f84fccacdde76c2b52de33 [term 1 LEADER]: Becoming Leader. State: Replica: 46ae53b938f84fccacdde76c2b52de33, State: Running, Role: LEADER
I20260811 02:56:41.524237 6233 consensus_queue.cc:237] T 00000000000000000000000000000000 P 46ae53b938f84fccacdde76c2b52de33 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "46ae53b938f84fccacdde76c2b52de33" member_type: VOTER last_known_addr { host: "127.12.138.254" port: 39201 } }
I20260811 02:56:41.524858 6221 sys_catalog.cc:564] T 00000000000000000000000000000000 P 46ae53b938f84fccacdde76c2b52de33 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 02:56:41.526175 6234 sys_catalog.cc:455] T 00000000000000000000000000000000 P 46ae53b938f84fccacdde76c2b52de33 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 46ae53b938f84fccacdde76c2b52de33. Latest consensus state: current_term: 1 leader_uuid: "46ae53b938f84fccacdde76c2b52de33" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "46ae53b938f84fccacdde76c2b52de33" member_type: VOTER last_known_addr { host: "127.12.138.254" port: 39201 } } }
I20260811 02:56:41.526175 6235 sys_catalog.cc:455] T 00000000000000000000000000000000 P 46ae53b938f84fccacdde76c2b52de33 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "46ae53b938f84fccacdde76c2b52de33" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "46ae53b938f84fccacdde76c2b52de33" member_type: VOTER last_known_addr { host: "127.12.138.254" port: 39201 } } }
I20260811 02:56:41.526371 6235 sys_catalog.cc:458] T 00000000000000000000000000000000 P 46ae53b938f84fccacdde76c2b52de33 [sys.catalog]: This master's current role is: LEADER
I20260811 02:56:41.526647 6234 sys_catalog.cc:458] T 00000000000000000000000000000000 P 46ae53b938f84fccacdde76c2b52de33 [sys.catalog]: This master's current role is: LEADER
I20260811 02:56:41.526849 6243 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 02:56:41.529855 6243 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 02:56:41.535989 6243 catalog_manager.cc:1347] Generated new cluster ID: c472016d78d748acade993f07f8093cc
I20260811 02:56:41.536069 6243 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 02:56:41.548193 6243 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 02:56:41.548542 6243 catalog_manager.cc:1504] Loading token signing keys...
I20260811 02:56:41.556350 6243 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 46ae53b938f84fccacdde76c2b52de33: Generated new TSK 0
I20260811 02:56:41.557065 6243 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 02:56:41.565181 12843 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDawEwS/build/debug/bin/kudu with pid 6142
2026-08-11T02:56:41Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/17 (1574 ms)
[----------] 18 tests from AuthTokenIssuingTest (28565 ms total)
[----------] 2 tests from ConnectToFollowerMasterTest
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0
Loading random data
Initializing database '/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786416707705494-12843-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 02:56:41 dist-test-slave-9gcw krb5kdc[6261](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 02:56:41 dist-test-slave-9gcw krb5kdc[6261](info): set up 2 sockets
Aug 11 02:56:41 dist-test-slave-9gcw krb5kdc[6261](info): commencing operation
krb5kdc: starting...
W20260811 02:56:42.695302 12843 mini_kdc.cc:121] Time spent starting KDC: real 1.106s user 0.006s sys 0.002s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786416707705494-12843-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786416707705494-12843-0/krb5kdc/test-user.keytab.
Aug 11 02:56:42 dist-test-slave-9gcw krb5kdc[6261](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417002, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T02:56:42Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T02:56:42Z Disabled control of system clock
WARNING: no policy specified for kudu/127.12.138.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.138.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.138.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786416707705494-12843-0/krb5kdc/kudu_127.12.138.254.keytab.
Entry for principal kudu/127.12.138.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786416707705494-12843-0/krb5kdc/kudu_127.12.138.254.keytab.
I20260811 02:56:42.841369 12843 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDawEwS/build/debug/bin/kudu
/tmp/dist-test-taskDawEwS/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786416707705494-12843-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786416707705494-12843-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786416707705494-12843-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786416707705494-12843-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.138.254:37743
--webserver_interface=127.12.138.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786416707705494-12843-0/krb5kdc/kudu_127.12.138.254.keytab
--principal=kudu/127.12.138.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.138.212:37527
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.12.138.254:37743,127.12.138.253:37657,127.12.138.252:41053
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786416707705494-12843-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786416707705494-12843-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786416707705494-12843-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 02:56:42.967754 6275 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260811 02:56:42.968111 6275 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 02:56:42.968174 6275 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 02:56:42.968258 6275 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 02:56:42.972357 6275 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 02:56:42.972456 6275 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 02:56:42.972496 6275 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 02:56:42.972534 6275 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 02:56:42.972566 6275 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 02:56:42.977977 6275 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.138.212:37527
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786416707705494-12843-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786416707705494-12843-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.12.138.254:37743,127.12.138.253:37657,127.12.138.252:41053
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786416707705494-12843-0/krb5kdc/kudu_127.12.138.254.keytab
--principal=kudu/127.12.138.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.138.254:37743
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786416707705494-12843-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.12.138.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6275
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786416707705494-12843-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 02:09:49 UTC on ccycloud.quasar-fsnqqc.root.comops.site
build id 4579570
I20260811 02:56:42.979127 6275 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 02:56:42.980213 6275 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 02:56:42.989389 6281 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:56:42.989355 6283 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:56:42.989367 6280 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:56:42.989804 6275 server_base.cc:1034] running on GCE node
I20260811 02:56:42.990438 6275 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 02:56:42.991605 6275 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 02:56:42.992820 6275 hybrid_clock.cc:648] HybridClock initialized: now 1786417002992804 us; error 26 us; skew 500 ppm
Aug 11 02:56:42 dist-test-slave-9gcw krb5kdc[6261](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417002, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.138.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 02:56:42.996240 6275 init.cc:377] Logged in from keytab as kudu/127.12.138.254@KRBTEST.COM (short username kudu)
I20260811 02:56:42.997443 6275 webserver.cc:460] Webserver started at http://127.12.138.254:45557/ using document root <none> and password file <none>
I20260811 02:56:42.998140 6275 fs_manager.cc:359] Metadata directory not provided
I20260811 02:56:42.998255 6275 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 02:56:42.998540 6275 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 02:56:43.000921 6275 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786416707705494-12843-0/minicluster-data/master-0/data/instance:
uuid: "874e24a90a2446baa6f2f7eb70ca1724"
format_stamp: "Formatted at 2026-08-11 02:56:42 on dist-test-slave-9gcw"
server_key: "ef35cf3dcde23cbbaf95dd38c9028f46"
server_key_iv: "641262361d9cb87dfd01cf086233c3f5"
server_key_version: "encryptionkey@0"
I20260811 02:56:43.001677 6275 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786416707705494-12843-0/minicluster-data/master-0/wal/instance:
uuid: "874e24a90a2446baa6f2f7eb70ca1724"
format_stamp: "Formatted at 2026-08-11 02:56:42 on dist-test-slave-9gcw"
server_key: "ef35cf3dcde23cbbaf95dd38c9028f46"
server_key_iv: "641262361d9cb87dfd01cf086233c3f5"
server_key_version: "encryptionkey@0"
I20260811 02:56:43.006192 6275 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260811 02:56:43.009073 6291 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 02:56:43.010531 6275 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 02:56:43.010700 6275 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786416707705494-12843-0/minicluster-data/master-0/data,/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786416707705494-12843-0/minicluster-data/master-0/wal
uuid: "874e24a90a2446baa6f2f7eb70ca1724"
format_stamp: "Formatted at 2026-08-11 02:56:42 on dist-test-slave-9gcw"
server_key: "ef35cf3dcde23cbbaf95dd38c9028f46"
server_key_iv: "641262361d9cb87dfd01cf086233c3f5"
server_key_version: "encryptionkey@0"
I20260811 02:56:43.010850 6275 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786416707705494-12843-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786416707705494-12843-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786416707705494-12843-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 02:56:43.032670 6275 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 02:56:43.036118 6275 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 02:56:43.036356 6275 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 02:56:43.044973 6275 rpc_server.cc:307] RPC server started. Bound to: 127.12.138.254:37743
I20260811 02:56:43.045328 6350 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.138.254:37743 every 8 connection(s)
I20260811 02:56:43.046160 6275 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786416707705494-12843-0/minicluster-data/master-0/data/info.pb
I20260811 02:56:43.049399 6351 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 02:56:43.050259 12843 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDawEwS/build/debug/bin/kudu as pid 6275
I20260811 02:56:43.050421 12843 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786416707705494-12843-0/minicluster-data/master-0/wal/instance
I20260811 02:56:43.050755 12843 external_mini_cluster.cc:1428] Setting key c51fe517e7c8169185bff712e328a56c
I20260811 02:56:43.054942 6351 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.12.138.254" port: 37743 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.12.138.253@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.138.253@KRBTEST.COM" created.
Aug 11 02:56:43 dist-test-slave-9gcw krb5kdc[6261](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417002, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.138.254@KRBTEST.COM for kudu/127.12.138.254@KRBTEST.COM
I20260811 02:56:43.074033 6355 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:56:43.057761 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59714 (local address 127.12.138.254:37743)
0811 02:56:43.058268 (+ 507us) server_negotiation.cc:206] Beginning negotiation
0811 02:56:43.058290 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0811 02:56:43.058590 (+ 300us) server_negotiation.cc:407] Connection header received
0811 02:56:43.060071 (+ 1481us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:56:43.060079 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:56:43.060298 (+ 219us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:56:43.060506 (+ 208us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:56:43.061602 (+ 1096us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:56:43.062696 (+ 1094us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:56:43.064080 (+ 1384us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:56:43.064764 (+ 684us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:56:43.064947 (+ 183us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:56:43.068638 (+ 3691us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:56:43.068647 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:56:43.068664 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:56:43.068697 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:56:43.070708 (+ 2011us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:56:43.071268 (+ 560us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:56:43.071274 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:56:43.071275 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:56:43.071380 (+ 105us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:56:43.071710 (+ 330us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:56:43.071714 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:56:43.071715 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:56:43.072110 (+ 395us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:56:43.072264 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0811 02:56:43.072661 (+ 397us) server_negotiation.cc:299] Negotiation successful
0811 02:56:43.072932 (+ 271us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":3,"server-negotiator.queue_time_us":327,"thread_start_us":166,"threads_started":1}
I20260811 02:56:43.074033 6356 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:56:43.057762 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.138.254:37743 (local address 127.0.0.1:59714)
0811 02:56:43.058428 (+ 666us) negotiation.cc:107] Waiting for socket to connect
0811 02:56:43.058470 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0811 02:56:43.059521 (+ 1051us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:56:43.060508 (+ 987us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:56:43.060514 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:56:43.061059 (+ 545us) client_negotiation.cc:189] Negotiated authn=SASL
0811 02:56:43.061416 (+ 357us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:56:43.061437 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:56:43.062867 (+ 1430us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:56:43.062869 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:56:43.063913 (+ 1044us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:56:43.063927 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:56:43.065041 (+ 1114us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:56:43.065050 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:56:43.065319 (+ 269us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:56:43.066748 (+ 1429us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 02:56:43.066770 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0811 02:56:43.068464 (+ 1694us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 02:56:43.070845 (+ 2381us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:56:43.070855 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:56:43.070866 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:56:43.071127 (+ 261us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:56:43.071483 (+ 356us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:56:43.071488 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:56:43.071490 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:56:43.071610 (+ 120us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:56:43.072232 (+ 622us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 02:56:43.072239 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 02:56:43.072396 (+ 157us) client_negotiation.cc:769] Sending connection context
0811 02:56:43.072574 (+ 178us) client_negotiation.cc:240] Negotiation successful
0811 02:56:43.072987 (+ 413us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":472,"mutex_wait_us":114,"spinlock_wait_cycles":7936,"thread_start_us":184,"threads_started":1}
I20260811 02:56:43.076793 6351 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.12.138.253" port: 37657 } has no permanent_uuid. Determining permanent_uuid...
I20260811 02:56:43.077682 6356 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:56:43.077358 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.138.253:37657 (local address 127.0.0.1:58322)
0811 02:56:43.077515 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0811 02:56:43.077605 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.138.253:37657: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
W20260811 02:56:43.077908 6292 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.12.138.253:37657: connect: Connection refused (error 111)
I20260811 02:56:43.078845 6356 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:56:43.078628 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.138.253:37657 (local address 127.0.0.1:58334)
0811 02:56:43.078734 (+ 106us) negotiation.cc:107] Waiting for socket to connect
0811 02:56:43.078790 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.138.253:37657: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
W20260811 02:56:43.079046 6351 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.12.138.253:37657: Network error: Client connection negotiation failed: client connection to 127.12.138.253:37657: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.138.253 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786416707705494-12843-0/krb5kdc/kudu_127.12.138.253.keytab.
Entry for principal kudu/127.12.138.253 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786416707705494-12843-0/krb5kdc/kudu_127.12.138.253.keytab.
I20260811 02:56:43.082831 12843 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDawEwS/build/debug/bin/kudu
/tmp/dist-test-taskDawEwS/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786416707705494-12843-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786416707705494-12843-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786416707705494-12843-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786416707705494-12843-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.138.253:37657
--webserver_interface=127.12.138.253
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786416707705494-12843-0/krb5kdc/kudu_127.12.138.253.keytab
--principal=kudu/127.12.138.253
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.138.212:37527
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.12.138.254:37743,127.12.138.253:37657,127.12.138.252:41053
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786416707705494-12843-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786416707705494-12843-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786416707705494-12843-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 02:56:43.128347 6351 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.12.138.253" port: 37657 } attempt: 1
I20260811 02:56:43.129429 6356 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:56:43.129003 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.138.253:37657 (local address 127.0.0.1:58350)
0811 02:56:43.129281 (+ 278us) negotiation.cc:107] Waiting for socket to connect
0811 02:56:43.129345 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.138.253:37657: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":99}
I20260811 02:56:43.130285 6356 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:56:43.130036 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.138.253:37657 (local address 127.0.0.1:58356)
0811 02:56:43.130173 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0811 02:56:43.130225 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.138.253:37657: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":60}
W20260811 02:56:43.130499 6351 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.12.138.253:37657: Network error: Client connection negotiation failed: client connection to 127.12.138.253:37657: connect: Connection refused (error 111)
I20260811 02:56:43.198699 6351 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.12.138.253" port: 37657 } attempt: 2
I20260811 02:56:43.199673 6356 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:56:43.199351 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.138.253:37657 (local address 127.0.0.1:58360)
0811 02:56:43.199517 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0811 02:56:43.199585 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.138.253:37657: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":66}
I20260811 02:56:43.200429 6356 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:56:43.200213 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.138.253:37657 (local address 127.0.0.1:58366)
0811 02:56:43.200324 (+ 111us) negotiation.cc:107] Waiting for socket to connect
0811 02:56:43.200378 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.138.253:37657: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
W20260811 02:56:43.200551 6351 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.12.138.253:37657: Network error: Client connection negotiation failed: client connection to 127.12.138.253:37657: connect: Connection refused (error 111)
W20260811 02:56:43.204480 6361 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260811 02:56:43.204838 6361 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 02:56:43.204906 6361 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 02:56:43.204993 6361 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 02:56:43.209688 6361 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 02:56:43.209767 6361 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 02:56:43.209820 6361 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 02:56:43.209860 6361 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 02:56:43.209892 6361 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 02:56:43.215958 6361 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.138.212:37527
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786416707705494-12843-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786416707705494-12843-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.12.138.254:37743,127.12.138.253:37657,127.12.138.252:41053
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786416707705494-12843-0/krb5kdc/kudu_127.12.138.253.keytab
--principal=kudu/127.12.138.253
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.138.253:37657
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786416707705494-12843-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.12.138.253
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6361
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786416707705494-12843-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 02:09:49 UTC on ccycloud.quasar-fsnqqc.root.comops.site
build id 4579570
I20260811 02:56:43.217118 6361 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 02:56:43.218235 6361 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 02:56:43.227068 6368 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:56:43.227174 6361 server_base.cc:1034] running on GCE node
W20260811 02:56:43.227145 6370 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:56:43.227102 6367 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:56:43.227980 6361 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 02:56:43.229333 6361 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 02:56:43.230526 6361 hybrid_clock.cc:648] HybridClock initialized: now 1786417003230493 us; error 47 us; skew 500 ppm
Aug 11 02:56:43 dist-test-slave-9gcw krb5kdc[6261](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417003, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.138.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 02:56:43.233870 6361 init.cc:377] Logged in from keytab as kudu/127.12.138.253@KRBTEST.COM (short username kudu)
I20260811 02:56:43.234822 6361 webserver.cc:460] Webserver started at http://127.12.138.253:38615/ using document root <none> and password file <none>
I20260811 02:56:43.235410 6361 fs_manager.cc:359] Metadata directory not provided
I20260811 02:56:43.235478 6361 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 02:56:43.235664 6361 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 02:56:43.238072 6361 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786416707705494-12843-0/minicluster-data/master-1/data/instance:
uuid: "1b898dc5660f41e1b61f081d0e7d8c4f"
format_stamp: "Formatted at 2026-08-11 02:56:43 on dist-test-slave-9gcw"
server_key: "1aa96e1088b004a95d0e457f25082514"
server_key_iv: "758299a16932ab7c0471be1e1a46bf2d"
server_key_version: "encryptionkey@0"
I20260811 02:56:43.238660 6361 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786416707705494-12843-0/minicluster-data/master-1/wal/instance:
uuid: "1b898dc5660f41e1b61f081d0e7d8c4f"
format_stamp: "Formatted at 2026-08-11 02:56:43 on dist-test-slave-9gcw"
server_key: "1aa96e1088b004a95d0e457f25082514"
server_key_iv: "758299a16932ab7c0471be1e1a46bf2d"
server_key_version: "encryptionkey@0"
I20260811 02:56:43.242725 6361 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260811 02:56:43.245546 6377 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 02:56:43.247015 6361 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 02:56:43.247128 6361 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786416707705494-12843-0/minicluster-data/master-1/data,/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786416707705494-12843-0/minicluster-data/master-1/wal
uuid: "1b898dc5660f41e1b61f081d0e7d8c4f"
format_stamp: "Formatted at 2026-08-11 02:56:43 on dist-test-slave-9gcw"
server_key: "1aa96e1088b004a95d0e457f25082514"
server_key_iv: "758299a16932ab7c0471be1e1a46bf2d"
server_key_version: "encryptionkey@0"
I20260811 02:56:43.247231 6361 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786416707705494-12843-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786416707705494-12843-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786416707705494-12843-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 02:56:43.274150 6361 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 02:56:43.278169 6361 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 02:56:43.278365 6361 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 02:56:43.286803 6361 rpc_server.cc:307] RPC server started. Bound to: 127.12.138.253:37657
I20260811 02:56:43.287112 6438 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.138.253:37657 every 8 connection(s)
I20260811 02:56:43.287981 6361 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786416707705494-12843-0/minicluster-data/master-1/data/info.pb
I20260811 02:56:43.290866 12843 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDawEwS/build/debug/bin/kudu as pid 6361
I20260811 02:56:43.291031 12843 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786416707705494-12843-0/minicluster-data/master-1/wal/instance
I20260811 02:56:43.291298 6439 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 02:56:43.291371 12843 external_mini_cluster.cc:1428] Setting key 3083443aa29a2e8377246f550f220f3e
I20260811 02:56:43.291742 6351 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.12.138.253" port: 37657 } attempt: 3
I20260811 02:56:43.298415 6439 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.12.138.254" port: 37743 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 02:56:43 dist-test-slave-9gcw krb5kdc[6261](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417002, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.138.254@KRBTEST.COM for kudu/127.12.138.253@KRBTEST.COM
WARNING: no policy specified for kudu/127.12.138.252@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.138.252@KRBTEST.COM" created.
I20260811 02:56:43.306969 6356 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:56:43.292393 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.138.253:37657 (local address 127.0.0.1:58374)
0811 02:56:43.292573 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0811 02:56:43.292596 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0811 02:56:43.292725 (+ 129us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:56:43.296019 (+ 3294us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:56:43.296023 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:56:43.296261 (+ 238us) client_negotiation.cc:189] Negotiated authn=SASL
0811 02:56:43.296420 (+ 159us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:56:43.296429 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:56:43.297921 (+ 1492us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:56:43.297926 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:56:43.298533 (+ 607us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:56:43.298543 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:56:43.299462 (+ 919us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:56:43.299467 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:56:43.299589 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:56:43.300198 (+ 609us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 02:56:43.300213 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0811 02:56:43.302089 (+ 1876us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 02:56:43.304647 (+ 2558us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:56:43.304651 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:56:43.304654 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:56:43.304891 (+ 237us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:56:43.305385 (+ 494us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:56:43.305388 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:56:43.305391 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:56:43.305493 (+ 102us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:56:43.306341 (+ 848us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 02:56:43.306345 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 02:56:43.306468 (+ 123us) client_negotiation.cc:769] Sending connection context
0811 02:56:43.306592 (+ 124us) client_negotiation.cc:240] Negotiation successful
0811 02:56:43.306769 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":49}
I20260811 02:56:43.308403 6441 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:56:43.293094 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58374 (local address 127.12.138.253:37657)
0811 02:56:43.293611 (+ 517us) server_negotiation.cc:206] Beginning negotiation
0811 02:56:43.293638 (+ 27us) server_negotiation.cc:399] Waiting for connection header
0811 02:56:43.293689 (+ 51us) server_negotiation.cc:407] Connection header received
0811 02:56:43.295166 (+ 1477us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:56:43.295199 (+ 33us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:56:43.295690 (+ 491us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:56:43.296087 (+ 397us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:56:43.296593 (+ 506us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:56:43.297772 (+ 1179us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:56:43.298672 (+ 900us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:56:43.299307 (+ 635us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:56:43.299499 (+ 192us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:56:43.302315 (+ 2816us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:56:43.302321 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:56:43.302331 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:56:43.302360 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:56:43.304521 (+ 2161us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:56:43.305021 (+ 500us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:56:43.305027 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:56:43.305034 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:56:43.305267 (+ 233us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:56:43.305601 (+ 334us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:56:43.305607 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:56:43.305609 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:56:43.306136 (+ 527us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:56:43.306364 (+ 228us) server_negotiation.cc:1035] Waiting for connection context
0811 02:56:43.306782 (+ 418us) server_negotiation.cc:299] Negotiation successful
0811 02:56:43.307130 (+ 348us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":361,"thread_start_us":148,"threads_started":1}
Aug 11 02:56:43 dist-test-slave-9gcw krb5kdc[6261](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417003, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.138.253@KRBTEST.COM for kudu/127.12.138.254@KRBTEST.COM
I20260811 02:56:43.310719 6351 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.12.138.252" port: 41053 } has no permanent_uuid. Determining permanent_uuid...
I20260811 02:56:43.311651 6356 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:56:43.311305 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.138.252:41053 (local address 127.0.0.1:52418)
0811 02:56:43.311477 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0811 02:56:43.311550 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.138.252:41053: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":64}
I20260811 02:56:43.312543 6356 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:56:43.312204 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.138.252:41053 (local address 127.0.0.1:52428)
0811 02:56:43.312423 (+ 219us) negotiation.cc:107] Waiting for socket to connect
0811 02:56:43.312476 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.138.252:41053: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":136}
W20260811 02:56:43.312716 6351 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.12.138.252:41053: Network error: Client connection negotiation failed: client connection to 127.12.138.252:41053: connect: Connection refused (error 111)
I20260811 02:56:43.314126 6355 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:56:43.300818 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59720 (local address 127.12.138.254:37743)
0811 02:56:43.300957 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0811 02:56:43.300962 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 02:56:43.301409 (+ 447us) server_negotiation.cc:407] Connection header received
0811 02:56:43.301674 (+ 265us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:56:43.301678 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:56:43.301742 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:56:43.301807 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:56:43.303091 (+ 1284us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:56:43.303970 (+ 879us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:56:43.304928 (+ 958us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:56:43.305947 (+ 1019us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:56:43.306082 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:56:43.309123 (+ 3041us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:56:43.309126 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:56:43.309130 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:56:43.309190 (+ 60us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:56:43.311757 (+ 2567us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:56:43.312446 (+ 689us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:56:43.312454 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:56:43.312458 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:56:43.312561 (+ 103us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:56:43.312919 (+ 358us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:56:43.312922 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:56:43.312923 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:56:43.313122 (+ 199us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:56:43.313286 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0811 02:56:43.313857 (+ 571us) server_negotiation.cc:299] Negotiation successful
0811 02:56:43.314007 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260811 02:56:43.314208 6443 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:56:43.300758 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.138.254:37743 (local address 127.0.0.1:59720)
0811 02:56:43.301268 (+ 510us) negotiation.cc:107] Waiting for socket to connect
0811 02:56:43.301326 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0811 02:56:43.301525 (+ 199us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:56:43.302074 (+ 549us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:56:43.302080 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:56:43.302688 (+ 608us) client_negotiation.cc:189] Negotiated authn=SASL
0811 02:56:43.302929 (+ 241us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:56:43.302942 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:56:43.304152 (+ 1210us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:56:43.304162 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:56:43.304781 (+ 619us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:56:43.304791 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:56:43.306233 (+ 1442us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:56:43.306237 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:56:43.306406 (+ 169us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:56:43.307071 (+ 665us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 02:56:43.307087 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0811 02:56:43.308951 (+ 1864us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 02:56:43.311903 (+ 2952us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:56:43.311911 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:56:43.311915 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:56:43.312275 (+ 360us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:56:43.312708 (+ 433us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:56:43.312715 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:56:43.312719 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:56:43.312828 (+ 109us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:56:43.313302 (+ 474us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 02:56:43.313311 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 02:56:43.313479 (+ 168us) client_negotiation.cc:769] Sending connection context
0811 02:56:43.313673 (+ 194us) client_negotiation.cc:240] Negotiation successful
0811 02:56:43.313952 (+ 279us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":294,"thread_start_us":197,"threads_started":1}
I20260811 02:56:43.315541 6439 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.12.138.253" port: 37657 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 02:56:43 dist-test-slave-9gcw krb5kdc[6261](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417003, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.138.253@KRBTEST.COM for kudu/127.12.138.253@KRBTEST.COM
I20260811 02:56:43.325085 6443 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:56:43.316094 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.138.253:37657 (local address 127.0.0.1:58378)
0811 02:56:43.316213 (+ 119us) negotiation.cc:107] Waiting for socket to connect
0811 02:56:43.316227 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 02:56:43.316312 (+ 85us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:56:43.316743 (+ 431us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:56:43.316745 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:56:43.316897 (+ 152us) client_negotiation.cc:189] Negotiated authn=SASL
0811 02:56:43.316998 (+ 101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:56:43.317003 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:56:43.317936 (+ 933us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:56:43.317939 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:56:43.318600 (+ 661us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:56:43.318610 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:56:43.319191 (+ 581us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:56:43.319194 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:56:43.319275 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:56:43.319810 (+ 535us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 02:56:43.319823 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0811 02:56:43.321304 (+ 1481us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 02:56:43.323371 (+ 2067us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:56:43.323373 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:56:43.323375 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:56:43.323578 (+ 203us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:56:43.323970 (+ 392us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:56:43.323975 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:56:43.323978 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:56:43.324093 (+ 115us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:56:43.324500 (+ 407us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 02:56:43.324504 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 02:56:43.324713 (+ 209us) client_negotiation.cc:769] Sending connection context
0811 02:56:43.324827 (+ 114us) client_negotiation.cc:240] Negotiation successful
0811 02:56:43.324960 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":49}
I20260811 02:56:43.325299 6441 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:56:43.316285 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58378 (local address 127.12.138.253:37657)
0811 02:56:43.316425 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0811 02:56:43.316432 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 02:56:43.316448 (+ 16us) server_negotiation.cc:407] Connection header received
0811 02:56:43.316490 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:56:43.316495 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:56:43.316538 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:56:43.316651 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:56:43.317122 (+ 471us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:56:43.317831 (+ 709us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:56:43.318745 (+ 914us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:56:43.319077 (+ 332us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:56:43.319167 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:56:43.321452 (+ 2285us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:56:43.321455 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:56:43.321457 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:56:43.321514 (+ 57us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:56:43.323242 (+ 1728us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:56:43.323740 (+ 498us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:56:43.323743 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:56:43.323744 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:56:43.323855 (+ 111us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:56:43.324192 (+ 337us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:56:43.324195 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:56:43.324196 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:56:43.324365 (+ 169us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:56:43.324443 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0811 02:56:43.324896 (+ 453us) server_negotiation.cc:299] Negotiation successful
0811 02:56:43.325020 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":18,"server-negotiator.queue_time_us":52}
I20260811 02:56:43.326371 6439 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.12.138.252" port: 41053 } has no permanent_uuid. Determining permanent_uuid...
I20260811 02:56:43.327416 6443 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:56:43.327034 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.138.252:41053 (local address 127.0.0.1:52434)
0811 02:56:43.327201 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0811 02:56:43.327298 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.138.252:41053: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":87}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.138.252 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786416707705494-12843-0/krb5kdc/kudu_127.12.138.252.keytab.
Entry for principal kudu/127.12.138.252 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786416707705494-12843-0/krb5kdc/kudu_127.12.138.252.keytab.
W20260811 02:56:43.327718 6383 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.12.138.252:41053: connect: Connection refused (error 111)
I20260811 02:56:43.328729 6443 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:56:43.328499 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.138.252:41053 (local address 127.0.0.1:52442)
0811 02:56:43.328632 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0811 02:56:43.328677 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.138.252:41053: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
I20260811 02:56:43.328841 12843 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDawEwS/build/debug/bin/kudu
/tmp/dist-test-taskDawEwS/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786416707705494-12843-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786416707705494-12843-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786416707705494-12843-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786416707705494-12843-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.138.252:41053
--webserver_interface=127.12.138.252
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786416707705494-12843-0/krb5kdc/kudu_127.12.138.252.keytab
--principal=kudu/127.12.138.252
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.138.212:37527
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.12.138.254:37743,127.12.138.253:37657,127.12.138.252:41053
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786416707705494-12843-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786416707705494-12843-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786416707705494-12843-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 02:56:43.328912 6439 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.12.138.252:41053: Network error: Client connection negotiation failed: client connection to 127.12.138.252:41053: connect: Connection refused (error 111)
I20260811 02:56:43.343904 6351 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.12.138.252" port: 41053 } attempt: 1
I20260811 02:56:43.344947 6356 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:56:43.344527 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.138.252:41053 (local address 127.0.0.1:52456)
0811 02:56:43.344783 (+ 256us) negotiation.cc:107] Waiting for socket to connect
0811 02:56:43.344862 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.138.252:41053: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":158}
I20260811 02:56:43.345969 6356 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:56:43.345694 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.138.252:41053 (local address 127.0.0.1:52464)
0811 02:56:43.345839 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0811 02:56:43.345899 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.138.252:41053: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":62}
W20260811 02:56:43.346160 6351 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.12.138.252:41053: Network error: Client connection negotiation failed: client connection to 127.12.138.252:41053: connect: Connection refused (error 111)
I20260811 02:56:43.378230 6439 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.12.138.252" port: 41053 } attempt: 1
I20260811 02:56:43.379184 6443 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:56:43.378866 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.138.252:41053 (local address 127.0.0.1:52476)
0811 02:56:43.379029 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0811 02:56:43.379098 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.138.252:41053: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":61}
I20260811 02:56:43.379951 6443 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:56:43.379734 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.138.252:41053 (local address 127.0.0.1:52478)
0811 02:56:43.379846 (+ 112us) negotiation.cc:107] Waiting for socket to connect
0811 02:56:43.379900 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.138.252:41053: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
W20260811 02:56:43.380082 6439 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.12.138.252:41053: Network error: Client connection negotiation failed: client connection to 127.12.138.252:41053: connect: Connection refused (error 111)
I20260811 02:56:43.421344 6351 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.12.138.252" port: 41053 } attempt: 2
I20260811 02:56:43.422219 6356 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:56:43.421899 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.138.252:41053 (local address 127.0.0.1:52480)
0811 02:56:43.422070 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0811 02:56:43.422134 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.138.252:41053: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":71}
I20260811 02:56:43.423142 6356 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:56:43.422851 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.138.252:41053 (local address 127.0.0.1:52490)
0811 02:56:43.423033 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0811 02:56:43.423081 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.138.252:41053: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":120}
W20260811 02:56:43.423333 6351 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.12.138.252:41053: Network error: Client connection negotiation failed: client connection to 127.12.138.252:41053: connect: Connection refused (error 111)
W20260811 02:56:43.447352 6447 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260811 02:56:43.447651 6447 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 02:56:43.447677 6447 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 02:56:43.447781 6447 flags.cc:425] Enabled unsafe flag: --never_fsync=true
I20260811 02:56:43.448254 6439 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.12.138.252" port: 41053 } attempt: 2
I20260811 02:56:43.449115 6443 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:56:43.448804 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.138.252:41053 (local address 127.0.0.1:52496)
0811 02:56:43.448970 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0811 02:56:43.449038 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.138.252:41053: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":63}
I20260811 02:56:43.449949 6443 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:56:43.449713 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.138.252:41053 (local address 127.0.0.1:52502)
0811 02:56:43.449844 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0811 02:56:43.449894 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.138.252:41053: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":65}
W20260811 02:56:43.450227 6439 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.12.138.252:41053: Network error: Client connection negotiation failed: client connection to 127.12.138.252:41053: connect: Connection refused (error 111)
W20260811 02:56:43.452348 6447 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 02:56:43.452404 6447 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 02:56:43.452423 6447 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 02:56:43.452481 6447 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 02:56:43.452535 6447 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 02:56:43.458420 6447 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.138.212:37527
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786416707705494-12843-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786416707705494-12843-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.12.138.254:37743,127.12.138.253:37657,127.12.138.252:41053
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786416707705494-12843-0/krb5kdc/kudu_127.12.138.252.keytab
--principal=kudu/127.12.138.252
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.138.252:41053
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786416707705494-12843-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.12.138.252
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6447
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786416707705494-12843-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 02:09:49 UTC on ccycloud.quasar-fsnqqc.root.comops.site
build id 4579570
I20260811 02:56:43.459568 6447 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 02:56:43.460649 6447 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 02:56:43.469575 6452 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:56:43.469642 6455 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:56:43.469555 6453 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:56:43.470085 6447 server_base.cc:1034] running on GCE node
I20260811 02:56:43.470731 6447 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 02:56:43.471957 6447 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 02:56:43.473229 6447 hybrid_clock.cc:648] HybridClock initialized: now 1786417003473207 us; error 46 us; skew 500 ppm
Aug 11 02:56:43 dist-test-slave-9gcw krb5kdc[6261](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417003, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.138.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 02:56:43.476426 6447 init.cc:377] Logged in from keytab as kudu/127.12.138.252@KRBTEST.COM (short username kudu)
I20260811 02:56:43.477566 6447 webserver.cc:460] Webserver started at http://127.12.138.252:37289/ using document root <none> and password file <none>
I20260811 02:56:43.478174 6447 fs_manager.cc:359] Metadata directory not provided
I20260811 02:56:43.478269 6447 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 02:56:43.478516 6447 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 02:56:43.480690 6447 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786416707705494-12843-0/minicluster-data/master-2/data/instance:
uuid: "975f768401b44c2a83fbb504316273dc"
format_stamp: "Formatted at 2026-08-11 02:56:43 on dist-test-slave-9gcw"
server_key: "01c35046eff47be85c5091fab74521ff"
server_key_iv: "09f9b5a1cf25afa7d99aed998a1e9129"
server_key_version: "encryptionkey@0"
I20260811 02:56:43.481364 6447 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786416707705494-12843-0/minicluster-data/master-2/wal/instance:
uuid: "975f768401b44c2a83fbb504316273dc"
format_stamp: "Formatted at 2026-08-11 02:56:43 on dist-test-slave-9gcw"
server_key: "01c35046eff47be85c5091fab74521ff"
server_key_iv: "09f9b5a1cf25afa7d99aed998a1e9129"
server_key_version: "encryptionkey@0"
I20260811 02:56:43.485388 6447 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260811 02:56:43.488215 6463 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 02:56:43.489737 6447 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 02:56:43.489888 6447 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786416707705494-12843-0/minicluster-data/master-2/data,/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786416707705494-12843-0/minicluster-data/master-2/wal
uuid: "975f768401b44c2a83fbb504316273dc"
format_stamp: "Formatted at 2026-08-11 02:56:43 on dist-test-slave-9gcw"
server_key: "01c35046eff47be85c5091fab74521ff"
server_key_iv: "09f9b5a1cf25afa7d99aed998a1e9129"
server_key_version: "encryptionkey@0"
I20260811 02:56:43.490015 6447 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786416707705494-12843-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786416707705494-12843-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786416707705494-12843-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 02:56:43.519570 6447 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 02:56:43.522534 6351 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.12.138.252" port: 41053 } attempt: 3
I20260811 02:56:43.522975 6447 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 02:56:43.523193 6447 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 02:56:43.523449 6356 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:56:43.523091 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.138.252:41053 (local address 127.0.0.1:52512)
0811 02:56:43.523284 (+ 193us) negotiation.cc:107] Waiting for socket to connect
0811 02:56:43.523363 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.138.252:41053: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":101}
I20260811 02:56:43.524283 6356 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:56:43.524018 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.138.252:41053 (local address 127.0.0.1:52528)
0811 02:56:43.524137 (+ 119us) negotiation.cc:107] Waiting for socket to connect
0811 02:56:43.524198 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.138.252:41053: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
W20260811 02:56:43.524468 6351 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.12.138.252:41053: Network error: Client connection negotiation failed: client connection to 127.12.138.252:41053: connect: Connection refused (error 111)
I20260811 02:56:43.531131 6447 rpc_server.cc:307] RPC server started. Bound to: 127.12.138.252:41053
I20260811 02:56:43.531453 6525 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.138.252:41053 every 8 connection(s)
I20260811 02:56:43.532152 6447 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786416707705494-12843-0/minicluster-data/master-2/data/info.pb
I20260811 02:56:43.535107 6526 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 02:56:43.537447 12843 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDawEwS/build/debug/bin/kudu as pid 6447
I20260811 02:56:43.537583 12843 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786416707705494-12843-0/minicluster-data/master-2/wal/instance
I20260811 02:56:43.537910 12843 external_mini_cluster.cc:1428] Setting key 2be97a6cc5de51c2767abbd09d6f0bd5
I20260811 02:56:43.539732 6526 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.12.138.254" port: 37743 } has no permanent_uuid. Determining permanent_uuid...
I20260811 02:56:43.541384 6439 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.12.138.252" port: 41053 } attempt: 3
Aug 11 02:56:43 dist-test-slave-9gcw krb5kdc[6261](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417003, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.138.252@KRBTEST.COM for kudu/127.12.138.254@KRBTEST.COM
Aug 11 02:56:43 dist-test-slave-9gcw krb5kdc[6261](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417003, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.138.253@KRBTEST.COM for kudu/127.12.138.252@KRBTEST.COM
Aug 11 02:56:43 dist-test-slave-9gcw krb5kdc[6261](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417002, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.12.138.252@KRBTEST.COM
I20260811 02:56:43.554277 6355 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:56:43.541484 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59736 (local address 127.12.138.254:37743)
0811 02:56:43.541598 (+ 114us) server_negotiation.cc:206] Beginning negotiation
0811 02:56:43.541602 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 02:56:43.542092 (+ 490us) server_negotiation.cc:407] Connection header received
0811 02:56:43.542306 (+ 214us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:56:43.542310 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:56:43.542357 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:56:43.542409 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:56:43.543449 (+ 1040us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:56:43.544299 (+ 850us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:56:43.545785 (+ 1486us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:56:43.546140 (+ 355us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:56:43.546239 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:56:43.549081 (+ 2842us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:56:43.549086 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:56:43.549089 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:56:43.549119 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:56:43.551523 (+ 2404us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:56:43.552248 (+ 725us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:56:43.552253 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:56:43.552255 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:56:43.552362 (+ 107us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:56:43.552775 (+ 413us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:56:43.552778 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:56:43.552779 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:56:43.552985 (+ 206us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:56:43.553078 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0811 02:56:43.553865 (+ 787us) server_negotiation.cc:299] Negotiation successful
0811 02:56:43.554026 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260811 02:56:43.554970 6443 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:56:43.541822 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.138.252:41053 (local address 127.0.0.1:52544)
0811 02:56:43.542086 (+ 264us) negotiation.cc:107] Waiting for socket to connect
0811 02:56:43.542102 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 02:56:43.542185 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:56:43.543808 (+ 1623us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:56:43.543812 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:56:43.544017 (+ 205us) client_negotiation.cc:189] Negotiated authn=SASL
0811 02:56:43.544126 (+ 109us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:56:43.544134 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:56:43.545236 (+ 1102us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:56:43.545239 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:56:43.545908 (+ 669us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:56:43.545919 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:56:43.546561 (+ 642us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:56:43.546566 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:56:43.546700 (+ 134us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:56:43.547283 (+ 583us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 02:56:43.547301 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0811 02:56:43.549392 (+ 2091us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 02:56:43.552411 (+ 3019us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:56:43.552416 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:56:43.552420 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:56:43.552690 (+ 270us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:56:43.553801 (+ 1111us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:56:43.553804 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:56:43.553806 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:56:43.553902 (+ 96us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:56:43.554458 (+ 556us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 02:56:43.554462 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 02:56:43.554563 (+ 101us) client_negotiation.cc:769] Sending connection context
0811 02:56:43.554646 (+ 83us) client_negotiation.cc:240] Negotiation successful
0811 02:56:43.554815 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":159}
I20260811 02:56:43.555999 6529 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:56:43.539325 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52530 (local address 127.12.138.252:41053)
0811 02:56:43.539760 (+ 435us) server_negotiation.cc:206] Beginning negotiation
0811 02:56:43.539781 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0811 02:56:43.539825 (+ 44us) server_negotiation.cc:407] Connection header received
0811 02:56:43.540583 (+ 758us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:56:43.540603 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:56:43.540945 (+ 342us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:56:43.541371 (+ 426us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:56:43.542487 (+ 1116us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:56:43.543934 (+ 1447us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:56:43.545176 (+ 1242us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:56:43.545902 (+ 726us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:56:43.546090 (+ 188us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:56:43.550443 (+ 4353us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:56:43.550447 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:56:43.550450 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:56:43.550478 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:56:43.553442 (+ 2964us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:56:43.554122 (+ 680us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:56:43.554126 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:56:43.554128 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:56:43.554225 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:56:43.554605 (+ 380us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:56:43.554608 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:56:43.554611 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:56:43.554817 (+ 206us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:56:43.555059 (+ 242us) server_negotiation.cc:1035] Waiting for connection context
0811 02:56:43.555401 (+ 342us) server_negotiation.cc:299] Negotiation successful
0811 02:56:43.555720 (+ 319us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":217,"server-negotiator.queue_time_us":291,"spinlock_wait_cycles":5376,"thread_start_us":141,"threads_started":1}
I20260811 02:56:43.556006 6530 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:56:43.541429 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.138.254:37743 (local address 127.0.0.1:59736)
0811 02:56:43.541956 (+ 527us) negotiation.cc:107] Waiting for socket to connect
0811 02:56:43.542011 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0811 02:56:43.542184 (+ 173us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:56:43.542578 (+ 394us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:56:43.542584 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:56:43.543068 (+ 484us) client_negotiation.cc:189] Negotiated authn=SASL
0811 02:56:43.543258 (+ 190us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:56:43.543270 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:56:43.544454 (+ 1184us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:56:43.544457 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:56:43.545640 (+ 1183us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:56:43.545661 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:56:43.546393 (+ 732us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:56:43.546399 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:56:43.546531 (+ 132us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:56:43.547056 (+ 525us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 02:56:43.547072 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0811 02:56:43.548922 (+ 1850us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 02:56:43.551646 (+ 2724us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:56:43.551657 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:56:43.551668 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:56:43.552133 (+ 465us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:56:43.552446 (+ 313us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:56:43.552449 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:56:43.552450 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:56:43.552638 (+ 188us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:56:43.553115 (+ 477us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 02:56:43.553122 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 02:56:43.553526 (+ 404us) client_negotiation.cc:769] Sending connection context
0811 02:56:43.553767 (+ 241us) client_negotiation.cc:240] Negotiation successful
0811 02:56:43.554130 (+ 363us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":349,"mutex_wait_us":69,"spinlock_wait_cycles":768,"thread_start_us":116,"threads_started":1}
I20260811 02:56:43.556006 6531 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:56:43.542170 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52544 (local address 127.12.138.252:41053)
0811 02:56:43.543531 (+ 1361us) server_negotiation.cc:206] Beginning negotiation
0811 02:56:43.543539 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 02:56:43.543550 (+ 11us) server_negotiation.cc:407] Connection header received
0811 02:56:43.543580 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:56:43.543582 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:56:43.543628 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:56:43.543710 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:56:43.544263 (+ 553us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:56:43.545043 (+ 780us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:56:43.546044 (+ 1001us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:56:43.546381 (+ 337us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:56:43.546518 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:56:43.549556 (+ 3038us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:56:43.549562 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:56:43.549574 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:56:43.549622 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:56:43.552306 (+ 2684us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:56:43.553578 (+ 1272us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:56:43.553583 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:56:43.553585 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:56:43.553671 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:56:43.554019 (+ 348us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:56:43.554022 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:56:43.554024 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:56:43.554312 (+ 288us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:56:43.555266 (+ 954us) server_negotiation.cc:1035] Waiting for connection context
0811 02:56:43.555388 (+ 122us) server_negotiation.cc:299] Negotiation successful
0811 02:56:43.555602 (+ 214us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":51,"server-negotiator.queue_time_us":1297,"spinlock_wait_cycles":4864,"thread_start_us":87,"threads_started":1}
I20260811 02:56:43.557926 6526 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.12.138.253" port: 37657 } has no permanent_uuid. Determining permanent_uuid...
I20260811 02:56:43.560454 6439 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1b898dc5660f41e1b61f081d0e7d8c4f: Bootstrap starting.
I20260811 02:56:43.562870 6439 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1b898dc5660f41e1b61f081d0e7d8c4f: Neither blocks nor log segments found. Creating new log.
I20260811 02:56:43.563846 6439 log.cc:824] T 00000000000000000000000000000000 P 1b898dc5660f41e1b61f081d0e7d8c4f: Log is configured to *not* fsync() on all Append() calls
Aug 11 02:56:43 dist-test-slave-9gcw krb5kdc[6261](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417002, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.12.138.254@KRBTEST.COM
I20260811 02:56:43.566406 6439 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1b898dc5660f41e1b61f081d0e7d8c4f: No bootstrap required, opened a new log
Aug 11 02:56:43 dist-test-slave-9gcw krb5kdc[6261](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417003, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.138.252@KRBTEST.COM for kudu/127.12.138.253@KRBTEST.COM
I20260811 02:56:43.570235 6355 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:56:43.558911 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59750 (local address 127.12.138.254:37743)
0811 02:56:43.559025 (+ 114us) server_negotiation.cc:206] Beginning negotiation
0811 02:56:43.559029 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 02:56:43.559206 (+ 177us) server_negotiation.cc:407] Connection header received
0811 02:56:43.559379 (+ 173us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:56:43.559387 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:56:43.559429 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:56:43.559521 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:56:43.560226 (+ 705us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:56:43.560836 (+ 610us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:56:43.561796 (+ 960us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:56:43.562042 (+ 246us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:56:43.562166 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:56:43.565744 (+ 3578us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:56:43.565748 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:56:43.565751 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:56:43.565780 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:56:43.567753 (+ 1973us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:56:43.568267 (+ 514us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:56:43.568271 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:56:43.568273 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:56:43.568363 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:56:43.569019 (+ 656us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:56:43.569023 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:56:43.569025 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:56:43.569295 (+ 270us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:56:43.569516 (+ 221us) server_negotiation.cc:1035] Waiting for connection context
0811 02:56:43.569913 (+ 397us) server_negotiation.cc:299] Negotiation successful
0811 02:56:43.570047 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260811 02:56:43.570793 6439 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1b898dc5660f41e1b61f081d0e7d8c4f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "874e24a90a2446baa6f2f7eb70ca1724" member_type: VOTER last_known_addr { host: "127.12.138.254" port: 37743 } } peers { permanent_uuid: "1b898dc5660f41e1b61f081d0e7d8c4f" member_type: VOTER last_known_addr { host: "127.12.138.253" port: 37657 } } peers { permanent_uuid: "975f768401b44c2a83fbb504316273dc" member_type: VOTER last_known_addr { host: "127.12.138.252" port: 41053 } }
I20260811 02:56:43.571060 6439 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1b898dc5660f41e1b61f081d0e7d8c4f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1b898dc5660f41e1b61f081d0e7d8c4f, State: Initialized, Role: FOLLOWER
I20260811 02:56:43.571422 6439 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1b898dc5660f41e1b61f081d0e7d8c4f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "874e24a90a2446baa6f2f7eb70ca1724" member_type: VOTER last_known_addr { host: "127.12.138.254" port: 37743 } } peers { permanent_uuid: "1b898dc5660f41e1b61f081d0e7d8c4f" member_type: VOTER last_known_addr { host: "127.12.138.253" port: 37657 } } peers { permanent_uuid: "975f768401b44c2a83fbb504316273dc" member_type: VOTER last_known_addr { host: "127.12.138.252" port: 41053 } }
I20260811 02:56:43.571929 6441 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:56:43.563083 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58386 (local address 127.12.138.253:37657)
0811 02:56:43.563255 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0811 02:56:43.563261 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 02:56:43.563278 (+ 17us) server_negotiation.cc:407] Connection header received
0811 02:56:43.563324 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:56:43.563330 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:56:43.563397 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:56:43.563510 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:56:43.564128 (+ 618us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:56:43.564790 (+ 662us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:56:43.565763 (+ 973us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:56:43.565994 (+ 231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:56:43.566124 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:56:43.568257 (+ 2133us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:56:43.568262 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:56:43.568274 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:56:43.568296 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:56:43.569986 (+ 1690us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:56:43.570467 (+ 481us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:56:43.570470 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:56:43.570472 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:56:43.570573 (+ 101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:56:43.570998 (+ 425us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:56:43.571000 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:56:43.571001 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:56:43.571250 (+ 249us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:56:43.571359 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0811 02:56:43.571677 (+ 318us) server_negotiation.cc:299] Negotiation successful
0811 02:56:43.571802 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260811 02:56:43.571923 6530 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:56:43.558623 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.138.253:37657 (local address 127.0.0.1:58386)
0811 02:56:43.558759 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0811 02:56:43.558778 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0811 02:56:43.558859 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:56:43.563552 (+ 4693us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:56:43.563555 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:56:43.563768 (+ 213us) client_negotiation.cc:189] Negotiated authn=SASL
0811 02:56:43.563904 (+ 136us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:56:43.563913 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:56:43.564920 (+ 1007us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:56:43.564924 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:56:43.565624 (+ 700us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:56:43.565636 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:56:43.566258 (+ 622us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:56:43.566263 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:56:43.566391 (+ 128us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:56:43.566867 (+ 476us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 02:56:43.566885 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0811 02:56:43.568121 (+ 1236us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 02:56:43.570116 (+ 1995us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:56:43.570119 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:56:43.570121 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:56:43.570366 (+ 245us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:56:43.570673 (+ 307us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:56:43.570676 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:56:43.570678 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:56:43.570788 (+ 110us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:56:43.571410 (+ 622us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 02:56:43.571413 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 02:56:43.571508 (+ 95us) client_negotiation.cc:769] Sending connection context
0811 02:56:43.571622 (+ 114us) client_negotiation.cc:240] Negotiation successful
0811 02:56:43.571761 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":48}
I20260811 02:56:43.572358 6533 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1b898dc5660f41e1b61f081d0e7d8c4f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "874e24a90a2446baa6f2f7eb70ca1724" member_type: VOTER last_known_addr { host: "127.12.138.254" port: 37743 } } peers { permanent_uuid: "1b898dc5660f41e1b61f081d0e7d8c4f" member_type: VOTER last_known_addr { host: "127.12.138.253" port: 37657 } } peers { permanent_uuid: "975f768401b44c2a83fbb504316273dc" member_type: VOTER last_known_addr { host: "127.12.138.252" port: 41053 } } }
I20260811 02:56:43.572512 6533 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1b898dc5660f41e1b61f081d0e7d8c4f [sys.catalog]: This master's current role is: FOLLOWER
I20260811 02:56:43.572757 6526 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.12.138.252" port: 41053 } has no permanent_uuid. Determining permanent_uuid...
I20260811 02:56:43.572969 6439 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1b898dc5660f41e1b61f081d0e7d8c4f [sys.catalog]: configured and running, proceeding with master startup.
Aug 11 02:56:43 dist-test-slave-9gcw krb5kdc[6261](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417003, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.138.252@KRBTEST.COM for kudu/127.12.138.252@KRBTEST.COM
W20260811 02:56:43.578516 6546 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 1b898dc5660f41e1b61f081d0e7d8c4f: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 02:56:43.578593 6546 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
Aug 11 02:56:43 dist-test-slave-9gcw krb5kdc[6261](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417002, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.12.138.253@KRBTEST.COM
I20260811 02:56:43.576944 6361 master_runner.cc:186] Error getting master registration for 127.12.138.254:37743: OK, instance_id { permanent_uuid: "874e24a90a2446baa6f2f7eb70ca1724" instance_seqno: 1786417003042662 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260811 02:56:43.581379 6361 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260811 02:56:43.583016 6529 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:56:43.573368 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52556 (local address 127.12.138.252:41053)
0811 02:56:43.573483 (+ 115us) server_negotiation.cc:206] Beginning negotiation
0811 02:56:43.573487 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 02:56:43.573614 (+ 127us) server_negotiation.cc:407] Connection header received
0811 02:56:43.573738 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:56:43.573741 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:56:43.573783 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:56:43.573872 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:56:43.574260 (+ 388us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:56:43.574811 (+ 551us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:56:43.575865 (+ 1054us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:56:43.576164 (+ 299us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:56:43.576236 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:56:43.579426 (+ 3190us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:56:43.579430 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:56:43.579433 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:56:43.579457 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:56:43.581109 (+ 1652us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:56:43.581764 (+ 655us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:56:43.581770 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:56:43.581773 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:56:43.581856 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:56:43.582139 (+ 283us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:56:43.582143 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:56:43.582144 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:56:43.582364 (+ 220us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:56:43.582472 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0811 02:56:43.582720 (+ 248us) server_negotiation.cc:299] Negotiation successful
0811 02:56:43.582890 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":70,"server-negotiator.queue_time_us":46}
I20260811 02:56:43.583016 6530 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:56:43.573486 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.138.252:41053 (local address 127.0.0.1:52556)
0811 02:56:43.573555 (+ 69us) negotiation.cc:107] Waiting for socket to connect
0811 02:56:43.573567 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 02:56:43.573665 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:56:43.573882 (+ 217us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:56:43.573884 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:56:43.574053 (+ 169us) client_negotiation.cc:189] Negotiated authn=SASL
0811 02:56:43.574134 (+ 81us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:56:43.574140 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:56:43.574935 (+ 795us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:56:43.574939 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:56:43.575745 (+ 806us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:56:43.575756 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:56:43.576406 (+ 650us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:56:43.576409 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:56:43.576478 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:56:43.577164 (+ 686us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 02:56:43.577180 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0811 02:56:43.579276 (+ 2096us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 02:56:43.581378 (+ 2102us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:56:43.581381 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:56:43.581383 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:56:43.581647 (+ 264us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:56:43.581957 (+ 310us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:56:43.581960 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:56:43.581962 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:56:43.582042 (+ 80us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:56:43.582472 (+ 430us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 02:56:43.582474 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 02:56:43.582566 (+ 92us) client_negotiation.cc:769] Sending connection context
0811 02:56:43.582675 (+ 109us) client_negotiation.cc:240] Negotiation successful
0811 02:56:43.582830 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":24}
I20260811 02:56:43.585592 6441 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:56:43.571854 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58396 (local address 127.12.138.253:37657)
0811 02:56:43.572150 (+ 296us) server_negotiation.cc:206] Beginning negotiation
0811 02:56:43.572155 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 02:56:43.572876 (+ 721us) server_negotiation.cc:407] Connection header received
0811 02:56:43.573026 (+ 150us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:56:43.573029 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:56:43.573078 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:56:43.573201 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:56:43.574020 (+ 819us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:56:43.574777 (+ 757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:56:43.576127 (+ 1350us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:56:43.576424 (+ 297us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:56:43.576500 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:56:43.580475 (+ 3975us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:56:43.580484 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:56:43.580487 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:56:43.580513 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:56:43.582245 (+ 1732us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:56:43.582787 (+ 542us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:56:43.582790 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:56:43.582793 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:56:43.582886 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:56:43.583256 (+ 370us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:56:43.583260 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:56:43.583262 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:56:43.583500 (+ 238us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:56:43.583669 (+ 169us) server_negotiation.cc:1035] Waiting for connection context
0811 02:56:43.585310 (+ 1641us) server_negotiation.cc:299] Negotiation successful
0811 02:56:43.585442 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":216,"thread_start_us":91,"threads_started":1}
I20260811 02:56:43.586508 6526 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 975f768401b44c2a83fbb504316273dc: Bootstrap starting.
I20260811 02:56:43.588904 6526 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 975f768401b44c2a83fbb504316273dc: Neither blocks nor log segments found. Creating new log.
I20260811 02:56:43.589111 12843 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 02:56:43.589730 6526 log.cc:824] T 00000000000000000000000000000000 P 975f768401b44c2a83fbb504316273dc: Log is configured to *not* fsync() on all Append() calls
I20260811 02:56:43.591630 6526 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 975f768401b44c2a83fbb504316273dc: No bootstrap required, opened a new log
I20260811 02:56:43.590617 6323 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "874e24a90a2446baa6f2f7eb70ca1724"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59750
I20260811 02:56:43.594974 6526 raft_consensus.cc:348] T 00000000000000000000000000000000 P 975f768401b44c2a83fbb504316273dc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "874e24a90a2446baa6f2f7eb70ca1724" member_type: VOTER last_known_addr { host: "127.12.138.254" port: 37743 } } peers { permanent_uuid: "1b898dc5660f41e1b61f081d0e7d8c4f" member_type: VOTER last_known_addr { host: "127.12.138.253" port: 37657 } } peers { permanent_uuid: "975f768401b44c2a83fbb504316273dc" member_type: VOTER last_known_addr { host: "127.12.138.252" port: 41053 } }
I20260811 02:56:43.595170 6526 raft_consensus.cc:729] T 00000000000000000000000000000000 P 975f768401b44c2a83fbb504316273dc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 975f768401b44c2a83fbb504316273dc, State: Initialized, Role: FOLLOWER
I20260811 02:56:43.595446 6526 consensus_queue.cc:260] T 00000000000000000000000000000000 P 975f768401b44c2a83fbb504316273dc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "874e24a90a2446baa6f2f7eb70ca1724" member_type: VOTER last_known_addr { host: "127.12.138.254" port: 37743 } } peers { permanent_uuid: "1b898dc5660f41e1b61f081d0e7d8c4f" member_type: VOTER last_known_addr { host: "127.12.138.253" port: 37657 } } peers { permanent_uuid: "975f768401b44c2a83fbb504316273dc" member_type: VOTER last_known_addr { host: "127.12.138.252" port: 41053 } }
I20260811 02:56:43.595886 6549 sys_catalog.cc:455] T 00000000000000000000000000000000 P 975f768401b44c2a83fbb504316273dc [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "874e24a90a2446baa6f2f7eb70ca1724" member_type: VOTER last_known_addr { host: "127.12.138.254" port: 37743 } } peers { permanent_uuid: "1b898dc5660f41e1b61f081d0e7d8c4f" member_type: VOTER last_known_addr { host: "127.12.138.253" port: 37657 } } peers { permanent_uuid: "975f768401b44c2a83fbb504316273dc" member_type: VOTER last_known_addr { host: "127.12.138.252" port: 41053 } } }
I20260811 02:56:43.596012 6549 sys_catalog.cc:458] T 00000000000000000000000000000000 P 975f768401b44c2a83fbb504316273dc [sys.catalog]: This master's current role is: FOLLOWER
I20260811 02:56:43.596661 6526 sys_catalog.cc:564] T 00000000000000000000000000000000 P 975f768401b44c2a83fbb504316273dc [sys.catalog]: configured and running, proceeding with master startup.
W20260811 02:56:43.601706 6563 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 975f768401b44c2a83fbb504316273dc: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 02:56:43.601788 6563 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 02:56:43.600291 6447 master_runner.cc:186] Error getting master registration for 127.12.138.254:37743: OK, instance_id { permanent_uuid: "874e24a90a2446baa6f2f7eb70ca1724" instance_seqno: 1786417003042662 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260811 02:56:43.603121 6447 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260811 02:56:43.688664 6351 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.12.138.252" port: 41053 } attempt: 4
Aug 11 02:56:43 dist-test-slave-9gcw krb5kdc[6261](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417002, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.138.254@KRBTEST.COM for kudu/127.12.138.252@KRBTEST.COM
I20260811 02:56:43.699137 6356 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:56:43.689477 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.138.252:41053 (local address 127.0.0.1:52562)
0811 02:56:43.689697 (+ 220us) negotiation.cc:107] Waiting for socket to connect
0811 02:56:43.689714 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 02:56:43.689837 (+ 123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:56:43.690116 (+ 279us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:56:43.690119 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:56:43.690376 (+ 257us) client_negotiation.cc:189] Negotiated authn=SASL
0811 02:56:43.690506 (+ 130us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:56:43.690513 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:56:43.691285 (+ 772us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:56:43.691289 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:56:43.692141 (+ 852us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:56:43.692151 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:56:43.692762 (+ 611us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:56:43.692766 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:56:43.692849 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:56:43.693476 (+ 627us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 02:56:43.693492 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0811 02:56:43.695292 (+ 1800us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 02:56:43.697402 (+ 2110us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:56:43.697406 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:56:43.697408 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:56:43.697667 (+ 259us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:56:43.697994 (+ 327us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:56:43.697997 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:56:43.697999 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:56:43.698087 (+ 88us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:56:43.698577 (+ 490us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 02:56:43.698580 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 02:56:43.698682 (+ 102us) client_negotiation.cc:769] Sending connection context
0811 02:56:43.698797 (+ 115us) client_negotiation.cc:240] Negotiation successful
0811 02:56:43.698942 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":146}
I20260811 02:56:43.699828 6529 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:56:43.689533 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52562 (local address 127.12.138.252:41053)
0811 02:56:43.689690 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0811 02:56:43.689705 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0811 02:56:43.689777 (+ 72us) server_negotiation.cc:407] Connection header received
0811 02:56:43.689934 (+ 157us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:56:43.689938 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:56:43.689984 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:56:43.690089 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:56:43.690633 (+ 544us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:56:43.691157 (+ 524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:56:43.692295 (+ 1138us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:56:43.692645 (+ 350us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:56:43.692744 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:56:43.695460 (+ 2716us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:56:43.695465 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:56:43.695468 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:56:43.695497 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:56:43.697270 (+ 1773us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:56:43.697799 (+ 529us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:56:43.697803 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:56:43.697805 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:56:43.697893 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:56:43.698184 (+ 291us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:56:43.698188 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:56:43.698190 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:56:43.698437 (+ 247us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:56:43.698531 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0811 02:56:43.699535 (+ 1004us) server_negotiation.cc:299] Negotiation successful
0811 02:56:43.699667 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260811 02:56:43.703476 6351 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 874e24a90a2446baa6f2f7eb70ca1724: Bootstrap starting.
I20260811 02:56:43.706589 6351 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 874e24a90a2446baa6f2f7eb70ca1724: Neither blocks nor log segments found. Creating new log.
I20260811 02:56:43.707665 6351 log.cc:824] T 00000000000000000000000000000000 P 874e24a90a2446baa6f2f7eb70ca1724: Log is configured to *not* fsync() on all Append() calls
I20260811 02:56:43.710145 6351 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 874e24a90a2446baa6f2f7eb70ca1724: No bootstrap required, opened a new log
I20260811 02:56:43.713378 6351 raft_consensus.cc:348] T 00000000000000000000000000000000 P 874e24a90a2446baa6f2f7eb70ca1724 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "874e24a90a2446baa6f2f7eb70ca1724" member_type: VOTER last_known_addr { host: "127.12.138.254" port: 37743 } } peers { permanent_uuid: "1b898dc5660f41e1b61f081d0e7d8c4f" member_type: VOTER last_known_addr { host: "127.12.138.253" port: 37657 } } peers { permanent_uuid: "975f768401b44c2a83fbb504316273dc" member_type: VOTER last_known_addr { host: "127.12.138.252" port: 41053 } }
I20260811 02:56:43.713672 6351 raft_consensus.cc:729] T 00000000000000000000000000000000 P 874e24a90a2446baa6f2f7eb70ca1724 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 874e24a90a2446baa6f2f7eb70ca1724, State: Initialized, Role: FOLLOWER
I20260811 02:56:43.714058 6351 consensus_queue.cc:260] T 00000000000000000000000000000000 P 874e24a90a2446baa6f2f7eb70ca1724 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "874e24a90a2446baa6f2f7eb70ca1724" member_type: VOTER last_known_addr { host: "127.12.138.254" port: 37743 } } peers { permanent_uuid: "1b898dc5660f41e1b61f081d0e7d8c4f" member_type: VOTER last_known_addr { host: "127.12.138.253" port: 37657 } } peers { permanent_uuid: "975f768401b44c2a83fbb504316273dc" member_type: VOTER last_known_addr { host: "127.12.138.252" port: 41053 } }
I20260811 02:56:43.714751 6567 sys_catalog.cc:455] T 00000000000000000000000000000000 P 874e24a90a2446baa6f2f7eb70ca1724 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "874e24a90a2446baa6f2f7eb70ca1724" member_type: VOTER last_known_addr { host: "127.12.138.254" port: 37743 } } peers { permanent_uuid: "1b898dc5660f41e1b61f081d0e7d8c4f" member_type: VOTER last_known_addr { host: "127.12.138.253" port: 37657 } } peers { permanent_uuid: "975f768401b44c2a83fbb504316273dc" member_type: VOTER last_known_addr { host: "127.12.138.252" port: 41053 } } }
I20260811 02:56:43.714969 6567 sys_catalog.cc:458] T 00000000000000000000000000000000 P 874e24a90a2446baa6f2f7eb70ca1724 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 02:56:43.715734 6351 sys_catalog.cc:564] T 00000000000000000000000000000000 P 874e24a90a2446baa6f2f7eb70ca1724 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 02:56:43.715785 6323 raft_consensus.cc:482] T 00000000000000000000000000000000 P 874e24a90a2446baa6f2f7eb70ca1724 [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260811 02:56:43.715917 6323 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 874e24a90a2446baa6f2f7eb70ca1724 [term 0 FOLLOWER]: Advancing to term 1
I20260811 02:56:43.717348 6323 raft_consensus.cc:504] T 00000000000000000000000000000000 P 874e24a90a2446baa6f2f7eb70ca1724 [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "874e24a90a2446baa6f2f7eb70ca1724" member_type: VOTER last_known_addr { host: "127.12.138.254" port: 37743 } } peers { permanent_uuid: "1b898dc5660f41e1b61f081d0e7d8c4f" member_type: VOTER last_known_addr { host: "127.12.138.253" port: 37657 } } peers { permanent_uuid: "975f768401b44c2a83fbb504316273dc" member_type: VOTER last_known_addr { host: "127.12.138.252" port: 41053 } }
I20260811 02:56:43.718379 6323 leader_election.cc:288] T 00000000000000000000000000000000 P 874e24a90a2446baa6f2f7eb70ca1724 [CANDIDATE]: Term 1 election: Requested vote from peers 1b898dc5660f41e1b61f081d0e7d8c4f (127.12.138.253:37657), 975f768401b44c2a83fbb504316273dc (127.12.138.252:41053)
I20260811 02:56:43.719120 6409 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "874e24a90a2446baa6f2f7eb70ca1724" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "1b898dc5660f41e1b61f081d0e7d8c4f"
I20260811 02:56:43.719255 6494 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "874e24a90a2446baa6f2f7eb70ca1724" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "975f768401b44c2a83fbb504316273dc"
I20260811 02:56:43.719391 6409 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1b898dc5660f41e1b61f081d0e7d8c4f [term 0 FOLLOWER]: Advancing to term 1
I20260811 02:56:43.719509 6494 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 975f768401b44c2a83fbb504316273dc [term 0 FOLLOWER]: Advancing to term 1
I20260811 02:56:43.720803 6409 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 1b898dc5660f41e1b61f081d0e7d8c4f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 874e24a90a2446baa6f2f7eb70ca1724 in term 1.
I20260811 02:56:43.720821 6494 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 975f768401b44c2a83fbb504316273dc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 874e24a90a2446baa6f2f7eb70ca1724 in term 1.
I20260811 02:56:43.721493 6292 leader_election.cc:302] T 00000000000000000000000000000000 P 874e24a90a2446baa6f2f7eb70ca1724 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1b898dc5660f41e1b61f081d0e7d8c4f, 874e24a90a2446baa6f2f7eb70ca1724; no voters:
I20260811 02:56:43.724944 6567 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 874e24a90a2446baa6f2f7eb70ca1724 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 02:56:43.725236 6567 raft_consensus.cc:686] T 00000000000000000000000000000000 P 874e24a90a2446baa6f2f7eb70ca1724 [term 1 LEADER]: Becoming Leader. State: Replica: 874e24a90a2446baa6f2f7eb70ca1724, State: Running, Role: LEADER
I20260811 02:56:43.725636 6567 consensus_queue.cc:237] T 00000000000000000000000000000000 P 874e24a90a2446baa6f2f7eb70ca1724 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "874e24a90a2446baa6f2f7eb70ca1724" member_type: VOTER last_known_addr { host: "127.12.138.254" port: 37743 } } peers { permanent_uuid: "1b898dc5660f41e1b61f081d0e7d8c4f" member_type: VOTER last_known_addr { host: "127.12.138.253" port: 37657 } } peers { permanent_uuid: "975f768401b44c2a83fbb504316273dc" member_type: VOTER last_known_addr { host: "127.12.138.252" port: 41053 } }
I20260811 02:56:43.727898 6567 sys_catalog.cc:455] T 00000000000000000000000000000000 P 874e24a90a2446baa6f2f7eb70ca1724 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 874e24a90a2446baa6f2f7eb70ca1724. Latest consensus state: current_term: 1 leader_uuid: "874e24a90a2446baa6f2f7eb70ca1724" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "874e24a90a2446baa6f2f7eb70ca1724" member_type: VOTER last_known_addr { host: "127.12.138.254" port: 37743 } } peers { permanent_uuid: "1b898dc5660f41e1b61f081d0e7d8c4f" member_type: VOTER last_known_addr { host: "127.12.138.253" port: 37657 } } peers { permanent_uuid: "975f768401b44c2a83fbb504316273dc" member_type: VOTER last_known_addr { host: "127.12.138.252" port: 41053 } } }
I20260811 02:56:43.728015 6567 sys_catalog.cc:458] T 00000000000000000000000000000000 P 874e24a90a2446baa6f2f7eb70ca1724 [sys.catalog]: This master's current role is: LEADER
I20260811 02:56:43.730111 6275 master_runner.cc:263] No leader master found from master 874e24a90a2446baa6f2f7eb70ca1724
I20260811 02:56:43.730180 6275 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260811 02:56:43.732090 6578 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 874e24a90a2446baa6f2f7eb70ca1724: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 02:56:43.732177 6578 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 02:56:43.734122 6589 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 02:56:43.735081 6589 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 02:56:43.752367 6441 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:56:43.723324 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58406 (local address 127.12.138.253:37657)
0811 02:56:43.723448 (+ 124us) server_negotiation.cc:206] Beginning negotiation
0811 02:56:43.723453 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 02:56:43.741619 (+ 18166us) server_negotiation.cc:407] Connection header received
0811 02:56:43.741895 (+ 276us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:56:43.741898 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:56:43.741957 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:56:43.742014 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:56:43.743176 (+ 1162us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:56:43.743871 (+ 695us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:56:43.745050 (+ 1179us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:56:43.745452 (+ 402us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:56:43.745548 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:56:43.747568 (+ 2020us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:56:43.747572 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:56:43.747574 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:56:43.747617 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:56:43.749576 (+ 1959us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:56:43.750705 (+ 1129us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:56:43.750711 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:56:43.750716 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:56:43.750837 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:56:43.751202 (+ 365us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:56:43.751208 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:56:43.751210 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:56:43.751475 (+ 265us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:56:43.751592 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0811 02:56:43.752019 (+ 427us) server_negotiation.cc:299] Negotiation successful
0811 02:56:43.752153 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260811 02:56:43.754282 6494 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 975f768401b44c2a83fbb504316273dc [term 1 FOLLOWER]: Refusing update from remote peer 874e24a90a2446baa6f2f7eb70ca1724: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 02:56:43.754297 6409 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 1b898dc5660f41e1b61f081d0e7d8c4f [term 1 FOLLOWER]: Refusing update from remote peer 874e24a90a2446baa6f2f7eb70ca1724: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 02:56:43.755367 6355 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:56:43.724033 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59754 (local address 127.12.138.254:37743)
0811 02:56:43.724149 (+ 116us) server_negotiation.cc:206] Beginning negotiation
0811 02:56:43.724154 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 02:56:43.744369 (+ 20215us) server_negotiation.cc:407] Connection header received
0811 02:56:43.744541 (+ 172us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:56:43.744544 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:56:43.744592 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:56:43.744663 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:56:43.745874 (+ 1211us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:56:43.746639 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:56:43.747803 (+ 1164us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:56:43.748210 (+ 407us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:56:43.748315 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:56:43.750329 (+ 2014us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:56:43.750336 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:56:43.750339 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:56:43.750372 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:56:43.752175 (+ 1803us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:56:43.753883 (+ 1708us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:56:43.753887 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:56:43.753889 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:56:43.753983 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:56:43.754308 (+ 325us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:56:43.754314 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:56:43.754319 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:56:43.754538 (+ 219us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:56:43.754652 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0811 02:56:43.755025 (+ 373us) server_negotiation.cc:299] Negotiation successful
0811 02:56:43.755154 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260811 02:56:43.755304 6567 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 874e24a90a2446baa6f2f7eb70ca1724 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1b898dc5660f41e1b61f081d0e7d8c4f" member_type: VOTER last_known_addr { host: "127.12.138.253" port: 37657 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 02:56:43.755801 6567 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 874e24a90a2446baa6f2f7eb70ca1724 [LEADER]: Connected to new peer: Peer: permanent_uuid: "975f768401b44c2a83fbb504316273dc" member_type: VOTER last_known_addr { host: "127.12.138.252" port: 41053 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 02:56:43.760987 6533 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1b898dc5660f41e1b61f081d0e7d8c4f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 874e24a90a2446baa6f2f7eb70ca1724. Latest consensus state: current_term: 1 leader_uuid: "874e24a90a2446baa6f2f7eb70ca1724" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "874e24a90a2446baa6f2f7eb70ca1724" member_type: VOTER last_known_addr { host: "127.12.138.254" port: 37743 } } peers { permanent_uuid: "1b898dc5660f41e1b61f081d0e7d8c4f" member_type: VOTER last_known_addr { host: "127.12.138.253" port: 37657 } } peers { permanent_uuid: "975f768401b44c2a83fbb504316273dc" member_type: VOTER last_known_addr { host: "127.12.138.252" port: 41053 } } }
I20260811 02:56:43.760995 6549 sys_catalog.cc:455] T 00000000000000000000000000000000 P 975f768401b44c2a83fbb504316273dc [sys.catalog]: SysCatalogTable state changed. Reason: New leader 874e24a90a2446baa6f2f7eb70ca1724. Latest consensus state: current_term: 1 leader_uuid: "874e24a90a2446baa6f2f7eb70ca1724" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "874e24a90a2446baa6f2f7eb70ca1724" member_type: VOTER last_known_addr { host: "127.12.138.254" port: 37743 } } peers { permanent_uuid: "1b898dc5660f41e1b61f081d0e7d8c4f" member_type: VOTER last_known_addr { host: "127.12.138.253" port: 37657 } } peers { permanent_uuid: "975f768401b44c2a83fbb504316273dc" member_type: VOTER last_known_addr { host: "127.12.138.252" port: 41053 } } }
I20260811 02:56:43.761125 6533 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1b898dc5660f41e1b61f081d0e7d8c4f [sys.catalog]: This master's current role is: FOLLOWER
I20260811 02:56:43.761124 6549 sys_catalog.cc:458] T 00000000000000000000000000000000 P 975f768401b44c2a83fbb504316273dc [sys.catalog]: This master's current role is: FOLLOWER
I20260811 02:56:43.771620 6529 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:56:43.724655 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52570 (local address 127.12.138.252:41053)
0811 02:56:43.724784 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0811 02:56:43.724789 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 02:56:43.750179 (+ 25390us) server_negotiation.cc:407] Connection header received
0811 02:56:43.750390 (+ 211us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:56:43.750394 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:56:43.750450 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:56:43.750506 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:56:43.761734 (+ 11228us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:56:43.762537 (+ 803us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:56:43.763774 (+ 1237us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:56:43.764108 (+ 334us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:56:43.764196 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:56:43.766464 (+ 2268us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:56:43.766468 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:56:43.766471 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:56:43.766501 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:56:43.768373 (+ 1872us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:56:43.768915 (+ 542us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:56:43.768919 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:56:43.768921 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:56:43.769018 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:56:43.770520 (+ 1502us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:56:43.770524 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:56:43.770526 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:56:43.770767 (+ 241us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:56:43.770892 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0811 02:56:43.771306 (+ 414us) server_negotiation.cc:299] Negotiation successful
0811 02:56:43.771450 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
W20260811 02:56:43.774843 6471 tablet.cc:2369] T 00000000000000000000000000000000 P 975f768401b44c2a83fbb504316273dc: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 02:56:43.775398 6587 sys_catalog.cc:455] T 00000000000000000000000000000000 P 874e24a90a2446baa6f2f7eb70ca1724 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "874e24a90a2446baa6f2f7eb70ca1724" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "874e24a90a2446baa6f2f7eb70ca1724" member_type: VOTER last_known_addr { host: "127.12.138.254" port: 37743 } } peers { permanent_uuid: "1b898dc5660f41e1b61f081d0e7d8c4f" member_type: VOTER last_known_addr { host: "127.12.138.253" port: 37657 } } peers { permanent_uuid: "975f768401b44c2a83fbb504316273dc" member_type: VOTER last_known_addr { host: "127.12.138.252" port: 41053 } } }
I20260811 02:56:43.775590 6587 sys_catalog.cc:458] T 00000000000000000000000000000000 P 874e24a90a2446baa6f2f7eb70ca1724 [sys.catalog]: This master's current role is: LEADER
I20260811 02:56:43.776664 6549 sys_catalog.cc:455] T 00000000000000000000000000000000 P 975f768401b44c2a83fbb504316273dc [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "874e24a90a2446baa6f2f7eb70ca1724" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "874e24a90a2446baa6f2f7eb70ca1724" member_type: VOTER last_known_addr { host: "127.12.138.254" port: 37743 } } peers { permanent_uuid: "1b898dc5660f41e1b61f081d0e7d8c4f" member_type: VOTER last_known_addr { host: "127.12.138.253" port: 37657 } } peers { permanent_uuid: "975f768401b44c2a83fbb504316273dc" member_type: VOTER last_known_addr { host: "127.12.138.252" port: 41053 } } }
I20260811 02:56:43.776801 6549 sys_catalog.cc:458] T 00000000000000000000000000000000 P 975f768401b44c2a83fbb504316273dc [sys.catalog]: This master's current role is: FOLLOWER
I20260811 02:56:43.777220 6592 mvcc.cc:205] Tried to move back new op lower bound from 7317164047332069376 to 7317164047264116736. Current Snapshot: MvccSnapshot[applied={T|T < 7317164047332069376}]
I20260811 02:56:43.777581 6567 sys_catalog.cc:455] T 00000000000000000000000000000000 P 874e24a90a2446baa6f2f7eb70ca1724 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "874e24a90a2446baa6f2f7eb70ca1724" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "874e24a90a2446baa6f2f7eb70ca1724" member_type: VOTER last_known_addr { host: "127.12.138.254" port: 37743 } } peers { permanent_uuid: "1b898dc5660f41e1b61f081d0e7d8c4f" member_type: VOTER last_known_addr { host: "127.12.138.253" port: 37657 } } peers { permanent_uuid: "975f768401b44c2a83fbb504316273dc" member_type: VOTER last_known_addr { host: "127.12.138.252" port: 41053 } } }
I20260811 02:56:43.777697 6567 sys_catalog.cc:458] T 00000000000000000000000000000000 P 874e24a90a2446baa6f2f7eb70ca1724 [sys.catalog]: This master's current role is: LEADER
I20260811 02:56:43.777827 6595 mvcc.cc:205] Tried to move back new op lower bound from 7317164047332069376 to 7317164047264116736. Current Snapshot: MvccSnapshot[applied={T|T < 7317164047332069376}]
I20260811 02:56:43.779027 6533 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1b898dc5660f41e1b61f081d0e7d8c4f [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "874e24a90a2446baa6f2f7eb70ca1724" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "874e24a90a2446baa6f2f7eb70ca1724" member_type: VOTER last_known_addr { host: "127.12.138.254" port: 37743 } } peers { permanent_uuid: "1b898dc5660f41e1b61f081d0e7d8c4f" member_type: VOTER last_known_addr { host: "127.12.138.253" port: 37657 } } peers { permanent_uuid: "975f768401b44c2a83fbb504316273dc" member_type: VOTER last_known_addr { host: "127.12.138.252" port: 41053 } } }
I20260811 02:56:43.779155 6533 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1b898dc5660f41e1b61f081d0e7d8c4f [sys.catalog]: This master's current role is: FOLLOWER
I20260811 02:56:43.779503 6594 mvcc.cc:205] Tried to move back new op lower bound from 7317164047332069376 to 7317164047264116736. Current Snapshot: MvccSnapshot[applied={T|T < 7317164047332069376}]
I20260811 02:56:43.783463 6589 catalog_manager.cc:1347] Generated new cluster ID: a499c1cfa9584c78bc5ee4d41d13c4da
I20260811 02:56:43.783546 6589 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 02:56:43.813684 6589 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 02:56:43.814958 6589 catalog_manager.cc:1504] Loading token signing keys...
I20260811 02:56:43.834882 6589 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 874e24a90a2446baa6f2f7eb70ca1724: Generated new TSK 0
I20260811 02:56:43.835872 6589 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260811 02:56:43.862154 6441 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 02:56:43.860314 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58412 (local address 127.12.138.253:37657)
0811 02:56:43.860444 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0811 02:56:43.860448 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 02:56:43.860993 (+ 545us) server_negotiation.cc:407] Connection header received
0811 02:56:43.861221 (+ 228us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:56:43.861225 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:56:43.861287 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:56:43.861341 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:56:43.862007 (+ 666us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:58412: BlockingRecv error: recv got EOF from 127.0.0.1:58412 (error 108)
Metrics: {"server-negotiator.queue_time_us":54}
W20260811 02:56:43.862202 6355 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 02:56:43.860292 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59770 (local address 127.12.138.254:37743)
0811 02:56:43.860440 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0811 02:56:43.860446 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 02:56:43.860739 (+ 293us) server_negotiation.cc:407] Connection header received
0811 02:56:43.860991 (+ 252us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:56:43.860995 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:56:43.861055 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:56:43.861164 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:56:43.862027 (+ 863us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:59770: BlockingRecv error: recv got EOF from 127.0.0.1:59770 (error 108)
Metrics: {"server-negotiator.queue_time_us":62}
W20260811 02:56:43.862596 6529 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 02:56:43.860783 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52574 (local address 127.12.138.252:41053)
0811 02:56:43.860944 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0811 02:56:43.860951 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 02:56:43.861483 (+ 532us) server_negotiation.cc:407] Connection header received
0811 02:56:43.861705 (+ 222us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:56:43.861712 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:56:43.861772 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:56:43.861834 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:56:43.862449 (+ 615us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:52574: BlockingRecv error: recv got EOF from 127.0.0.1:52574 (error 108)
Metrics: {"server-negotiator.queue_time_us":58}
W20260811 02:56:43.866493 6441 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 02:56:43.865334 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58420 (local address 127.12.138.253:37657)
0811 02:56:43.865475 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0811 02:56:43.865478 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 02:56:43.865587 (+ 109us) server_negotiation.cc:407] Connection header received
0811 02:56:43.865746 (+ 159us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:56:43.865748 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:56:43.865817 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:56:43.865893 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:56:43.866377 (+ 484us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:58420: BlockingRecv error: recv got EOF from 127.0.0.1:58420 (error 108)
Metrics: {"server-negotiator.queue_time_us":57}
W20260811 02:56:43.870966 6441 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 02:56:43.870027 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58436 (local address 127.12.138.253:37657)
0811 02:56:43.870150 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0811 02:56:43.870155 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 02:56:43.870168 (+ 13us) server_negotiation.cc:407] Connection header received
0811 02:56:43.870213 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:56:43.870216 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:56:43.870274 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:56:43.870358 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:56:43.870842 (+ 484us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:58436: BlockingRecv error: recv got EOF from 127.0.0.1:58436 (error 108)
Metrics: {"server-negotiator.queue_time_us":52}
W20260811 02:56:43.876250 6441 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 02:56:43.875071 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58440 (local address 127.12.138.253:37657)
0811 02:56:43.875201 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0811 02:56:43.875205 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 02:56:43.875402 (+ 197us) server_negotiation.cc:407] Connection header received
0811 02:56:43.875506 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:56:43.875509 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:56:43.875556 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:56:43.875643 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:56:43.876144 (+ 501us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:58440: BlockingRecv error: recv got EOF from 127.0.0.1:58440 (error 108)
Metrics: {"server-negotiator.queue_time_us":50}
W20260811 02:56:43.883538 6441 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 02:56:43.882333 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58454 (local address 127.12.138.253:37657)
0811 02:56:43.882548 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0811 02:56:43.882552 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 02:56:43.882621 (+ 69us) server_negotiation.cc:407] Connection header received
0811 02:56:43.882845 (+ 224us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:56:43.882850 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:56:43.882914 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:56:43.882999 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:56:43.883427 (+ 428us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:58454: BlockingRecv error: recv got EOF from 127.0.0.1:58454 (error 108)
Metrics: {"server-negotiator.queue_time_us":138}
W20260811 02:56:43.895522 6441 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 02:56:43.893915 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58470 (local address 127.12.138.253:37657)
0811 02:56:43.894067 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0811 02:56:43.894073 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 02:56:43.894367 (+ 294us) server_negotiation.cc:407] Connection header received
0811 02:56:43.894609 (+ 242us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:56:43.894613 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:56:43.894677 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:56:43.894759 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:56:43.895349 (+ 590us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:58470: BlockingRecv error: recv got EOF from 127.0.0.1:58470 (error 108)
Metrics: {"server-negotiator.queue_time_us":42}
W20260811 02:56:43.915402 6441 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 02:56:43.913889 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58476 (local address 127.12.138.253:37657)
0811 02:56:43.914029 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0811 02:56:43.914034 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 02:56:43.914340 (+ 306us) server_negotiation.cc:407] Connection header received
0811 02:56:43.914568 (+ 228us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:56:43.914571 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:56:43.914624 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:56:43.914724 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:56:43.915250 (+ 526us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:58476: BlockingRecv error: recv got EOF from 127.0.0.1:58476 (error 108)
Metrics: {"server-negotiator.queue_time_us":56}
W20260811 02:56:43.953033 6441 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 02:56:43.951552 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58482 (local address 127.12.138.253:37657)
0811 02:56:43.951696 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0811 02:56:43.951701 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 02:56:43.951910 (+ 209us) server_negotiation.cc:407] Connection header received
0811 02:56:43.952127 (+ 217us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:56:43.952131 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:56:43.952193 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:56:43.952297 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:56:43.952903 (+ 606us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:58482: BlockingRecv error: recv got EOF from 127.0.0.1:58482 (error 108)
Metrics: {"server-negotiator.queue_time_us":58}
W20260811 02:56:44.021543 6441 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 02:56:44.020111 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58494 (local address 127.12.138.253:37657)
0811 02:56:44.020277 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0811 02:56:44.020283 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 02:56:44.020439 (+ 156us) server_negotiation.cc:407] Connection header received
0811 02:56:44.020567 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:56:44.020570 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:56:44.020620 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:56:44.020776 (+ 156us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:56:44.021379 (+ 603us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:58494: BlockingRecv error: recv got EOF from 127.0.0.1:58494 (error 108)
Metrics: {"server-negotiator.queue_time_us":68}
W20260811 02:56:44.153326 6441 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 02:56:44.151980 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58510 (local address 127.12.138.253:37657)
0811 02:56:44.152139 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0811 02:56:44.152144 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 02:56:44.152320 (+ 176us) server_negotiation.cc:407] Connection header received
0811 02:56:44.152433 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:56:44.152435 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:56:44.152485 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:56:44.152561 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:56:44.153144 (+ 583us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:58510: BlockingRecv error: recv got EOF from 127.0.0.1:58510 (error 108)
Metrics: {"server-negotiator.queue_time_us":61}
W20260811 02:56:44.412783 6441 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 02:56:44.411790 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58518 (local address 127.12.138.253:37657)
0811 02:56:44.411936 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0811 02:56:44.411940 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 02:56:44.411955 (+ 15us) server_negotiation.cc:407] Connection header received
0811 02:56:44.412029 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:56:44.412031 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:56:44.412077 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:56:44.412145 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:56:44.412656 (+ 511us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:58518: BlockingRecv error: recv got EOF from 127.0.0.1:58518 (error 108)
Metrics: {"server-negotiator.queue_time_us":64}
I20260811 02:56:44.580173 6546 catalog_manager.cc:1259] Loaded cluster ID: a499c1cfa9584c78bc5ee4d41d13c4da
I20260811 02:56:44.580272 6546 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 1b898dc5660f41e1b61f081d0e7d8c4f: loading cluster ID for follower catalog manager: success
I20260811 02:56:44.582314 6546 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 1b898dc5660f41e1b61f081d0e7d8c4f: acquiring CA information for follower catalog manager: success
I20260811 02:56:44.583804 6546 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 1b898dc5660f41e1b61f081d0e7d8c4f: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260811 02:56:44.603283 6563 catalog_manager.cc:1259] Loaded cluster ID: a499c1cfa9584c78bc5ee4d41d13c4da
I20260811 02:56:44.603384 6563 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 975f768401b44c2a83fbb504316273dc: loading cluster ID for follower catalog manager: success
I20260811 02:56:44.605301 6563 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 975f768401b44c2a83fbb504316273dc: acquiring CA information for follower catalog manager: success
I20260811 02:56:44.606603 6563 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 975f768401b44c2a83fbb504316273dc: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260811 02:56:44.932816 6679 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:56:44.927476 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58530 (local address 127.12.138.253:37657)
0811 02:56:44.927786 (+ 310us) server_negotiation.cc:206] Beginning negotiation
0811 02:56:44.927791 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 02:56:44.927827 (+ 36us) server_negotiation.cc:407] Connection header received
0811 02:56:44.927900 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:56:44.927902 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:56:44.928124 (+ 222us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:56:44.928277 (+ 153us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 02:56:44.928700 (+ 423us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:56:44.929671 (+ 971us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:56:44.931028 (+ 1357us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:56:44.931394 (+ 366us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:56:44.931516 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:56:44.931891 (+ 375us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 02:56:44.932389 (+ 498us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 02:56:44.932539 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0811 02:56:44.932578 (+ 39us) server_negotiation.cc:299] Negotiation successful
0811 02:56:44.932634 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":188,"thread_start_us":70,"threads_started":1}
I20260811 02:56:44.940711 6686 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:56:44.936550 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52586 (local address 127.12.138.252:41053)
0811 02:56:44.936843 (+ 293us) server_negotiation.cc:206] Beginning negotiation
0811 02:56:44.936851 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 02:56:44.936870 (+ 19us) server_negotiation.cc:407] Connection header received
0811 02:56:44.936979 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:56:44.936982 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:56:44.937103 (+ 121us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:56:44.937282 (+ 179us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 02:56:44.937537 (+ 255us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:56:44.938119 (+ 582us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:56:44.939069 (+ 950us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:56:44.939336 (+ 267us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:56:44.939461 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:56:44.939810 (+ 349us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 02:56:44.940272 (+ 462us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 02:56:44.940423 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0811 02:56:44.940471 (+ 48us) server_negotiation.cc:299] Negotiation successful
0811 02:56:44.940529 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":189,"thread_start_us":76,"threads_started":1}
I20260811 02:56:44.948866 6686 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:56:44.944712 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52590 (local address 127.12.138.252:41053)
0811 02:56:44.944853 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0811 02:56:44.944857 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 02:56:44.945545 (+ 688us) server_negotiation.cc:407] Connection header received
0811 02:56:44.945657 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:56:44.945660 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:56:44.945719 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:56:44.945794 (+ 75us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 02:56:44.946080 (+ 286us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:56:44.946660 (+ 580us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:56:44.947592 (+ 932us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:56:44.947965 (+ 373us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:56:44.948097 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:56:44.948341 (+ 244us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 02:56:44.948423 (+ 82us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 02:56:44.948501 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0811 02:56:44.948651 (+ 150us) server_negotiation.cc:299] Negotiation successful
0811 02:56:44.948718 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260811 02:56:44.949374 6693 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:56:44.944318 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59776 (local address 127.12.138.254:37743)
0811 02:56:44.944763 (+ 445us) server_negotiation.cc:206] Beginning negotiation
0811 02:56:44.944770 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 02:56:44.944785 (+ 15us) server_negotiation.cc:407] Connection header received
0811 02:56:44.944883 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:56:44.944886 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:56:44.945031 (+ 145us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:56:44.945190 (+ 159us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 02:56:44.945435 (+ 245us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:56:44.946222 (+ 787us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:56:44.947499 (+ 1277us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:56:44.947789 (+ 290us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:56:44.947902 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:56:44.948285 (+ 383us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 02:56:44.948676 (+ 391us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 02:56:44.949076 (+ 400us) server_negotiation.cc:1035] Waiting for connection context
0811 02:56:44.949118 (+ 42us) server_negotiation.cc:299] Negotiation successful
0811 02:56:44.949220 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":355,"thread_start_us":65,"threads_started":1}
I20260811 02:56:44.949795 6679 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:56:44.944460 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58546 (local address 127.12.138.253:37657)
0811 02:56:44.944603 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0811 02:56:44.944610 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 02:56:44.945216 (+ 606us) server_negotiation.cc:407] Connection header received
0811 02:56:44.945309 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:56:44.945312 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:56:44.945376 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:56:44.945462 (+ 86us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 02:56:44.945990 (+ 528us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:56:44.946837 (+ 847us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:56:44.948214 (+ 1377us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:56:44.948621 (+ 407us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:56:44.948728 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:56:44.949201 (+ 473us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 02:56:44.949289 (+ 88us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 02:56:44.949363 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0811 02:56:44.949595 (+ 232us) server_negotiation.cc:299] Negotiation successful
0811 02:56:44.949661 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260811 02:56:44.953346 12843 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDawEwS/build/debug/bin/kudu with pid 6275
I20260811 02:56:44.964020 12843 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDawEwS/build/debug/bin/kudu with pid 6361
I20260811 02:56:44.973291 12843 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDawEwS/build/debug/bin/kudu with pid 6447
2026-08-11T02:56:44Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0 (3408 ms)
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1
Loading random data
Initializing database '/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786416707705494-12843-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 02:56:45 dist-test-slave-9gcw krb5kdc[6698](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 02:56:45 dist-test-slave-9gcw krb5kdc[6698](info): set up 2 sockets
Aug 11 02:56:45 dist-test-slave-9gcw krb5kdc[6698](info): commencing operation
krb5kdc: starting...
W20260811 02:56:46.196372 12843 mini_kdc.cc:121] Time spent starting KDC: real 1.198s user 0.005s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786416707705494-12843-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786416707705494-12843-0/krb5kdc/test-user.keytab.
Aug 11 02:56:46 dist-test-slave-9gcw krb5kdc[6698](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417006, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T02:56:46Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T02:56:46Z Disabled control of system clock
WARNING: no policy specified for kudu/127.12.138.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.138.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.138.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786416707705494-12843-0/krb5kdc/kudu_127.12.138.254.keytab.
Entry for principal kudu/127.12.138.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786416707705494-12843-0/krb5kdc/kudu_127.12.138.254.keytab.
I20260811 02:56:46.337519 12843 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDawEwS/build/debug/bin/kudu
/tmp/dist-test-taskDawEwS/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786416707705494-12843-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786416707705494-12843-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786416707705494-12843-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786416707705494-12843-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.138.254:44727
--webserver_interface=127.12.138.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786416707705494-12843-0/krb5kdc/kudu_127.12.138.254.keytab
--principal=kudu/127.12.138.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.138.212:41187
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.12.138.254:44727,127.12.138.253:43519,127.12.138.252:44637
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786416707705494-12843-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786416707705494-12843-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786416707705494-12843-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 02:56:46.459337 6713 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260811 02:56:46.459725 6713 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 02:56:46.459805 6713 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 02:56:46.459887 6713 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 02:56:46.463960 6713 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 02:56:46.464071 6713 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 02:56:46.464109 6713 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 02:56:46.464148 6713 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 02:56:46.464184 6713 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 02:56:46.469676 6713 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.138.212:41187
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786416707705494-12843-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786416707705494-12843-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.12.138.254:44727,127.12.138.253:43519,127.12.138.252:44637
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786416707705494-12843-0/krb5kdc/kudu_127.12.138.254.keytab
--principal=kudu/127.12.138.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.138.254:44727
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786416707705494-12843-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.12.138.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6713
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786416707705494-12843-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 02:09:49 UTC on ccycloud.quasar-fsnqqc.root.comops.site
build id 4579570
I20260811 02:56:46.471004 6713 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 02:56:46.472086 6713 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 02:56:46.481122 6718 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:56:46.481138 6722 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:56:46.481122 6719 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:56:46.481204 6713 server_base.cc:1034] running on GCE node
I20260811 02:56:46.482103 6713 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 02:56:46.483362 6713 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 02:56:46.484570 6713 hybrid_clock.cc:648] HybridClock initialized: now 1786417006484514 us; error 70 us; skew 500 ppm
Aug 11 02:56:46 dist-test-slave-9gcw krb5kdc[6698](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417006, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.138.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 02:56:46.491303 6713 init.cc:377] Logged in from keytab as kudu/127.12.138.254@KRBTEST.COM (short username kudu)
I20260811 02:56:46.492388 6713 webserver.cc:460] Webserver started at http://127.12.138.254:44913/ using document root <none> and password file <none>
I20260811 02:56:46.493019 6713 fs_manager.cc:359] Metadata directory not provided
I20260811 02:56:46.493132 6713 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 02:56:46.493394 6713 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 02:56:46.495735 6713 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786416707705494-12843-0/minicluster-data/master-0/data/instance:
uuid: "71018b8f50c84cf4991af4f7257992a4"
format_stamp: "Formatted at 2026-08-11 02:56:46 on dist-test-slave-9gcw"
server_key: "258584ddb2ff2c414caa8f7e6aab33c1"
server_key_iv: "e08173fe2c37cd225eeedc2849a96a9c"
server_key_version: "encryptionkey@0"
I20260811 02:56:46.496414 6713 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786416707705494-12843-0/minicluster-data/master-0/wal/instance:
uuid: "71018b8f50c84cf4991af4f7257992a4"
format_stamp: "Formatted at 2026-08-11 02:56:46 on dist-test-slave-9gcw"
server_key: "258584ddb2ff2c414caa8f7e6aab33c1"
server_key_iv: "e08173fe2c37cd225eeedc2849a96a9c"
server_key_version: "encryptionkey@0"
I20260811 02:56:46.500527 6713 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.002s
I20260811 02:56:46.503388 6729 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 02:56:46.504788 6713 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260811 02:56:46.504966 6713 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786416707705494-12843-0/minicluster-data/master-0/data,/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786416707705494-12843-0/minicluster-data/master-0/wal
uuid: "71018b8f50c84cf4991af4f7257992a4"
format_stamp: "Formatted at 2026-08-11 02:56:46 on dist-test-slave-9gcw"
server_key: "258584ddb2ff2c414caa8f7e6aab33c1"
server_key_iv: "e08173fe2c37cd225eeedc2849a96a9c"
server_key_version: "encryptionkey@0"
I20260811 02:56:46.505093 6713 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786416707705494-12843-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786416707705494-12843-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786416707705494-12843-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 02:56:46.515640 6713 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 02:56:46.518893 6713 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 02:56:46.519099 6713 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 02:56:46.526863 6713 rpc_server.cc:307] RPC server started. Bound to: 127.12.138.254:44727
I20260811 02:56:46.527176 6796 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.138.254:44727 every 8 connection(s)
I20260811 02:56:46.527920 6713 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786416707705494-12843-0/minicluster-data/master-0/data/info.pb
I20260811 02:56:46.531219 6797 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 02:56:46.535615 6797 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.12.138.254" port: 44727 } has no permanent_uuid. Determining permanent_uuid...
I20260811 02:56:46.535944 12843 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDawEwS/build/debug/bin/kudu as pid 6713
I20260811 02:56:46.536090 12843 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786416707705494-12843-0/minicluster-data/master-0/wal/instance
I20260811 02:56:46.536381 12843 external_mini_cluster.cc:1428] Setting key 0fafaef798d5066b6680a554408119eb
WARNING: no policy specified for kudu/127.12.138.253@KRBTEST.COM; defaulting to no policy
Aug 11 02:56:46 dist-test-slave-9gcw krb5kdc[6698](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417006, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.138.254@KRBTEST.COM for kudu/127.12.138.254@KRBTEST.COM
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.138.253@KRBTEST.COM" created.
I20260811 02:56:46.553571 6801 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:56:46.538305 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35298 (local address 127.12.138.254:44727)
0811 02:56:46.538866 (+ 561us) server_negotiation.cc:206] Beginning negotiation
0811 02:56:46.538887 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0811 02:56:46.539323 (+ 436us) server_negotiation.cc:407] Connection header received
0811 02:56:46.540656 (+ 1333us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:56:46.540665 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:56:46.540930 (+ 265us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:56:46.541170 (+ 240us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:56:46.541987 (+ 817us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:56:46.543251 (+ 1264us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:56:46.544250 (+ 999us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:56:46.544639 (+ 389us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:56:46.544814 (+ 175us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:56:46.547819 (+ 3005us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:56:46.547826 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:56:46.547845 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:56:46.547866 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:56:46.549932 (+ 2066us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:56:46.550539 (+ 607us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:56:46.550554 (+ 15us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:56:46.550556 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:56:46.550674 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:56:46.551029 (+ 355us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:56:46.551034 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:56:46.551037 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:56:46.551498 (+ 461us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:56:46.551667 (+ 169us) server_negotiation.cc:1035] Waiting for connection context
0811 02:56:46.552078 (+ 411us) server_negotiation.cc:299] Negotiation successful
0811 02:56:46.552362 (+ 284us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":381,"spinlock_wait_cycles":57088,"thread_start_us":182,"threads_started":1}
I20260811 02:56:46.553571 6800 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:56:46.538305 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.138.254:44727 (local address 127.0.0.1:35298)
0811 02:56:46.539137 (+ 832us) negotiation.cc:107] Waiting for socket to connect
0811 02:56:46.539192 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0811 02:56:46.540302 (+ 1110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:56:46.541211 (+ 909us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:56:46.541216 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:56:46.541561 (+ 345us) client_negotiation.cc:189] Negotiated authn=SASL
0811 02:56:46.541826 (+ 265us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:56:46.541839 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:56:46.543379 (+ 1540us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:56:46.543382 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:56:46.544126 (+ 744us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:56:46.544135 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:56:46.544799 (+ 664us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:56:46.544803 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:56:46.544897 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:56:46.545740 (+ 843us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 02:56:46.545767 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0811 02:56:46.547670 (+ 1903us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 02:56:46.550056 (+ 2386us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:56:46.550067 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:56:46.550080 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:56:46.550414 (+ 334us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:56:46.550791 (+ 377us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:56:46.550794 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:56:46.550797 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:56:46.550925 (+ 128us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:56:46.551640 (+ 715us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 02:56:46.551648 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 02:56:46.551805 (+ 157us) client_negotiation.cc:769] Sending connection context
0811 02:56:46.551995 (+ 190us) client_negotiation.cc:240] Negotiation successful
0811 02:56:46.552425 (+ 430us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":638,"mutex_wait_us":120,"spinlock_wait_cycles":768,"thread_start_us":183,"threads_started":1}
I20260811 02:56:46.556309 6797 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.12.138.253" port: 43519 } has no permanent_uuid. Determining permanent_uuid...
I20260811 02:56:46.557331 6800 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:56:46.556916 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.138.253:43519 (local address 127.0.0.1:56504)
0811 02:56:46.557084 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0811 02:56:46.557230 (+ 146us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.138.253:43519: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
W20260811 02:56:46.557566 6736 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.12.138.253:43519: connect: Connection refused (error 111)
I20260811 02:56:46.558599 6800 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:56:46.558320 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.138.253:43519 (local address 127.0.0.1:56508)
0811 02:56:46.558462 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0811 02:56:46.558524 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.138.253:43519: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
W20260811 02:56:46.558804 6797 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.12.138.253:43519: Network error: Client connection negotiation failed: client connection to 127.12.138.253:43519: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.138.253 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786416707705494-12843-0/krb5kdc/kudu_127.12.138.253.keytab.
Entry for principal kudu/127.12.138.253 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786416707705494-12843-0/krb5kdc/kudu_127.12.138.253.keytab.
I20260811 02:56:46.569041 12843 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDawEwS/build/debug/bin/kudu
/tmp/dist-test-taskDawEwS/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786416707705494-12843-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786416707705494-12843-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786416707705494-12843-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786416707705494-12843-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.138.253:43519
--webserver_interface=127.12.138.253
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786416707705494-12843-0/krb5kdc/kudu_127.12.138.253.keytab
--principal=kudu/127.12.138.253
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.138.212:41187
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.12.138.254:44727,127.12.138.253:43519,127.12.138.252:44637
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786416707705494-12843-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786416707705494-12843-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786416707705494-12843-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 02:56:46.608079 6797 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.12.138.253" port: 43519 } attempt: 1
I20260811 02:56:46.609077 6800 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:56:46.608689 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.138.253:43519 (local address 127.0.0.1:56522)
0811 02:56:46.608874 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0811 02:56:46.608979 (+ 105us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.138.253:43519: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":64}
I20260811 02:56:46.609895 6800 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:56:46.609650 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.138.253:43519 (local address 127.0.0.1:56528)
0811 02:56:46.609773 (+ 123us) negotiation.cc:107] Waiting for socket to connect
0811 02:56:46.609829 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.138.253:43519: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
W20260811 02:56:46.610049 6797 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.12.138.253:43519: Network error: Client connection negotiation failed: client connection to 127.12.138.253:43519: connect: Connection refused (error 111)
I20260811 02:56:46.678234 6797 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.12.138.253" port: 43519 } attempt: 2
I20260811 02:56:46.679140 6800 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:56:46.678813 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.138.253:43519 (local address 127.0.0.1:56536)
0811 02:56:46.678980 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0811 02:56:46.679047 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.138.253:43519: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":66}
I20260811 02:56:46.679919 6800 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:56:46.679695 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.138.253:43519 (local address 127.0.0.1:56538)
0811 02:56:46.679814 (+ 119us) negotiation.cc:107] Waiting for socket to connect
0811 02:56:46.679868 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.138.253:43519: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
W20260811 02:56:46.680043 6797 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.12.138.253:43519: Network error: Client connection negotiation failed: client connection to 127.12.138.253:43519: connect: Connection refused (error 111)
W20260811 02:56:46.687012 6804 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260811 02:56:46.687325 6804 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 02:56:46.687386 6804 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 02:56:46.687463 6804 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 02:56:46.691550 6804 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 02:56:46.691638 6804 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 02:56:46.691686 6804 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 02:56:46.691738 6804 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 02:56:46.691776 6804 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 02:56:46.697190 6804 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.138.212:41187
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786416707705494-12843-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786416707705494-12843-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.12.138.254:44727,127.12.138.253:43519,127.12.138.252:44637
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786416707705494-12843-0/krb5kdc/kudu_127.12.138.253.keytab
--principal=kudu/127.12.138.253
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.138.253:43519
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786416707705494-12843-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.12.138.253
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6804
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786416707705494-12843-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 02:09:49 UTC on ccycloud.quasar-fsnqqc.root.comops.site
build id 4579570
I20260811 02:56:46.698312 6804 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 02:56:46.699347 6804 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 02:56:46.708112 6811 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:56:46.708114 6812 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:56:46.708160 6814 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:56:46.708427 6804 server_base.cc:1034] running on GCE node
I20260811 02:56:46.709004 6804 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 02:56:46.710274 6804 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 02:56:46.711465 6804 hybrid_clock.cc:648] HybridClock initialized: now 1786417006711452 us; error 26 us; skew 500 ppm
Aug 11 02:56:46 dist-test-slave-9gcw krb5kdc[6698](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417006, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.138.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 02:56:46.714398 6804 init.cc:377] Logged in from keytab as kudu/127.12.138.253@KRBTEST.COM (short username kudu)
I20260811 02:56:46.715265 6804 webserver.cc:460] Webserver started at http://127.12.138.253:40269/ using document root <none> and password file <none>
I20260811 02:56:46.715880 6804 fs_manager.cc:359] Metadata directory not provided
I20260811 02:56:46.715975 6804 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 02:56:46.716223 6804 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 02:56:46.718353 6804 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786416707705494-12843-0/minicluster-data/master-1/data/instance:
uuid: "9cc09b2071ea409587a0ceb8a1d7c94a"
format_stamp: "Formatted at 2026-08-11 02:56:46 on dist-test-slave-9gcw"
server_key: "c2ca9ec6b5fbe53c0730956d31194dc7"
server_key_iv: "8c2f6bdad9273a9b840a9562f6c4d29b"
server_key_version: "encryptionkey@0"
I20260811 02:56:46.718946 6804 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786416707705494-12843-0/minicluster-data/master-1/wal/instance:
uuid: "9cc09b2071ea409587a0ceb8a1d7c94a"
format_stamp: "Formatted at 2026-08-11 02:56:46 on dist-test-slave-9gcw"
server_key: "c2ca9ec6b5fbe53c0730956d31194dc7"
server_key_iv: "8c2f6bdad9273a9b840a9562f6c4d29b"
server_key_version: "encryptionkey@0"
I20260811 02:56:46.723359 6804 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260811 02:56:46.725921 6823 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 02:56:46.727174 6804 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260811 02:56:46.727325 6804 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786416707705494-12843-0/minicluster-data/master-1/data,/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786416707705494-12843-0/minicluster-data/master-1/wal
uuid: "9cc09b2071ea409587a0ceb8a1d7c94a"
format_stamp: "Formatted at 2026-08-11 02:56:46 on dist-test-slave-9gcw"
server_key: "c2ca9ec6b5fbe53c0730956d31194dc7"
server_key_iv: "8c2f6bdad9273a9b840a9562f6c4d29b"
server_key_version: "encryptionkey@0"
I20260811 02:56:46.727442 6804 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786416707705494-12843-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786416707705494-12843-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786416707705494-12843-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 02:56:46.751255 6804 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 02:56:46.754559 6804 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 02:56:46.754797 6804 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 02:56:46.762125 6804 rpc_server.cc:307] RPC server started. Bound to: 127.12.138.253:43519
I20260811 02:56:46.762450 6882 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.138.253:43519 every 8 connection(s)
I20260811 02:56:46.763155 6804 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786416707705494-12843-0/minicluster-data/master-1/data/info.pb
I20260811 02:56:46.766217 6883 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 02:56:46.766788 12843 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDawEwS/build/debug/bin/kudu as pid 6804
I20260811 02:56:46.766918 12843 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786416707705494-12843-0/minicluster-data/master-1/wal/instance
I20260811 02:56:46.767251 12843 external_mini_cluster.cc:1428] Setting key e8e0b4ec9fd1cf162d1abf471b3367ed
I20260811 02:56:46.771215 6797 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.12.138.253" port: 43519 } attempt: 3
I20260811 02:56:46.771171 6883 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.12.138.254" port: 44727 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.12.138.252@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.138.252@KRBTEST.COM" created.
Aug 11 02:56:46 dist-test-slave-9gcw krb5kdc[6698](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417006, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.138.254@KRBTEST.COM for kudu/127.12.138.253@KRBTEST.COM
Aug 11 02:56:46 dist-test-slave-9gcw krb5kdc[6698](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417006, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.138.253@KRBTEST.COM for kudu/127.12.138.254@KRBTEST.COM
I20260811 02:56:46.786787 6801 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:56:46.773257 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35308 (local address 127.12.138.254:44727)
0811 02:56:46.773360 (+ 103us) server_negotiation.cc:206] Beginning negotiation
0811 02:56:46.773364 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 02:56:46.773782 (+ 418us) server_negotiation.cc:407] Connection header received
0811 02:56:46.774362 (+ 580us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:56:46.774365 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:56:46.774415 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:56:46.774511 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:56:46.775826 (+ 1315us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:56:46.776666 (+ 840us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:56:46.778081 (+ 1415us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:56:46.778990 (+ 909us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:56:46.779103 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:56:46.782896 (+ 3793us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:56:46.782900 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:56:46.782902 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:56:46.782927 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:56:46.784476 (+ 1549us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:56:46.785182 (+ 706us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:56:46.785188 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:56:46.785190 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:56:46.785309 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:56:46.785659 (+ 350us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:56:46.785662 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:56:46.785663 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:56:46.785857 (+ 194us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:56:46.785971 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0811 02:56:46.786460 (+ 489us) server_negotiation.cc:299] Negotiation successful
0811 02:56:46.786596 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260811 02:56:46.787056 6800 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:56:46.771788 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.138.253:43519 (local address 127.0.0.1:56544)
0811 02:56:46.771998 (+ 210us) negotiation.cc:107] Waiting for socket to connect
0811 02:56:46.772015 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 02:56:46.772111 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:56:46.774571 (+ 2460us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:56:46.774576 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:56:46.774812 (+ 236us) client_negotiation.cc:189] Negotiated authn=SASL
0811 02:56:46.774983 (+ 171us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:56:46.774995 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:56:46.776517 (+ 1522us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:56:46.776522 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:56:46.777302 (+ 780us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:56:46.777314 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:56:46.778326 (+ 1012us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:56:46.778330 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:56:46.778444 (+ 114us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:56:46.779344 (+ 900us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 02:56:46.779355 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0811 02:56:46.782080 (+ 2725us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 02:56:46.784779 (+ 2699us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:56:46.784783 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:56:46.784786 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:56:46.784989 (+ 203us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:56:46.785422 (+ 433us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:56:46.785428 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:56:46.785432 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:56:46.785532 (+ 100us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:56:46.786385 (+ 853us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 02:56:46.786391 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 02:56:46.786523 (+ 132us) client_negotiation.cc:769] Sending connection context
0811 02:56:46.786613 (+ 90us) client_negotiation.cc:240] Negotiation successful
0811 02:56:46.786806 (+ 193us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":129}
I20260811 02:56:46.787830 6888 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:56:46.773204 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.138.254:44727 (local address 127.0.0.1:35308)
0811 02:56:46.773624 (+ 420us) negotiation.cc:107] Waiting for socket to connect
0811 02:56:46.773682 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0811 02:56:46.774078 (+ 396us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:56:46.774954 (+ 876us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:56:46.774958 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:56:46.775450 (+ 492us) client_negotiation.cc:189] Negotiated authn=SASL
0811 02:56:46.775640 (+ 190us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:56:46.775666 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:56:46.776816 (+ 1150us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:56:46.776820 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:56:46.777951 (+ 1131us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:56:46.777961 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:56:46.779167 (+ 1206us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:56:46.779172 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:56:46.779298 (+ 126us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:56:46.779842 (+ 544us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 02:56:46.779867 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0811 02:56:46.782737 (+ 2870us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 02:56:46.784615 (+ 1878us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:56:46.784626 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:56:46.784637 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:56:46.784990 (+ 353us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:56:46.785411 (+ 421us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:56:46.785413 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:56:46.785414 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:56:46.785534 (+ 120us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:56:46.785971 (+ 437us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 02:56:46.785980 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 02:56:46.786202 (+ 222us) client_negotiation.cc:769] Sending connection context
0811 02:56:46.786386 (+ 184us) client_negotiation.cc:240] Negotiation successful
0811 02:56:46.786723 (+ 337us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":229,"mutex_wait_us":120,"thread_start_us":142,"threads_started":1}
I20260811 02:56:46.787835 6887 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:56:46.772558 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56544 (local address 127.12.138.253:43519)
0811 02:56:46.773083 (+ 525us) server_negotiation.cc:206] Beginning negotiation
0811 02:56:46.773102 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0811 02:56:46.773141 (+ 39us) server_negotiation.cc:407] Connection header received
0811 02:56:46.774070 (+ 929us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:56:46.774090 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:56:46.774384 (+ 294us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:56:46.774561 (+ 177us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:56:46.775142 (+ 581us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:56:46.776391 (+ 1249us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:56:46.777433 (+ 1042us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:56:46.778198 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:56:46.778323 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:56:46.782268 (+ 3945us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:56:46.782280 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:56:46.782296 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:56:46.782326 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:56:46.784674 (+ 2348us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:56:46.785109 (+ 435us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:56:46.785119 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:56:46.785123 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:56:46.785299 (+ 176us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:56:46.785652 (+ 353us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:56:46.785658 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:56:46.785662 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:56:46.786241 (+ 579us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:56:46.787202 (+ 961us) server_negotiation.cc:1035] Waiting for connection context
0811 02:56:46.787323 (+ 121us) server_negotiation.cc:299] Negotiation successful
0811 02:56:46.787498 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":56,"server-negotiator.queue_time_us":349,"thread_start_us":155,"threads_started":1}
I20260811 02:56:46.789312 6883 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.12.138.253" port: 43519 } has no permanent_uuid. Determining permanent_uuid...
I20260811 02:56:46.790077 6797 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.12.138.252" port: 44637 } has no permanent_uuid. Determining permanent_uuid...
I20260811 02:56:46.790874 6800 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:56:46.790559 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.138.252:44637 (local address 127.0.0.1:50286)
0811 02:56:46.790742 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0811 02:56:46.790784 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.138.252:44637: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":117}
I20260811 02:56:46.791589 6800 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:56:46.791334 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.138.252:44637 (local address 127.0.0.1:50288)
0811 02:56:46.791482 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0811 02:56:46.791529 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.138.252:44637: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":81}
W20260811 02:56:46.791712 6797 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.12.138.252:44637: Network error: Client connection negotiation failed: client connection to 127.12.138.252:44637: connect: Connection refused (error 111)
Aug 11 02:56:46 dist-test-slave-9gcw krb5kdc[6698](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417006, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.138.253@KRBTEST.COM for kudu/127.12.138.253@KRBTEST.COM
I20260811 02:56:46.798949 6887 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:56:46.790123 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56552 (local address 127.12.138.253:43519)
0811 02:56:46.790256 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0811 02:56:46.790260 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 02:56:46.790576 (+ 316us) server_negotiation.cc:407] Connection header received
0811 02:56:46.790684 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:56:46.790688 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:56:46.790742 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:56:46.790827 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:56:46.791305 (+ 478us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:56:46.792290 (+ 985us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:56:46.793090 (+ 800us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:56:46.793485 (+ 395us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:56:46.793567 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:56:46.795632 (+ 2065us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:56:46.795636 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:56:46.795638 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:56:46.795661 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:56:46.797246 (+ 1585us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:56:46.797688 (+ 442us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:56:46.797691 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:56:46.797693 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:56:46.797771 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:56:46.798061 (+ 290us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:56:46.798066 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:56:46.798068 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:56:46.798299 (+ 231us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:56:46.798399 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0811 02:56:46.798649 (+ 250us) server_negotiation.cc:299] Negotiation successful
0811 02:56:46.798797 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":35,"server-negotiator.queue_time_us":52}
I20260811 02:56:46.798974 6888 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:56:46.789883 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.138.253:43519 (local address 127.0.0.1:56552)
0811 02:56:46.790484 (+ 601us) negotiation.cc:107] Waiting for socket to connect
0811 02:56:46.790499 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 02:56:46.790588 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:56:46.790861 (+ 273us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:56:46.790864 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:56:46.791045 (+ 181us) client_negotiation.cc:189] Negotiated authn=SASL
0811 02:56:46.791164 (+ 119us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:56:46.791198 (+ 34us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:56:46.792430 (+ 1232us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:56:46.792432 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:56:46.792977 (+ 545us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:56:46.792987 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:56:46.793579 (+ 592us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:56:46.793582 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:56:46.793649 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:56:46.794159 (+ 510us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 02:56:46.794189 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0811 02:56:46.795438 (+ 1249us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 02:56:46.797348 (+ 1910us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:56:46.797353 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:56:46.797355 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:56:46.797571 (+ 216us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:56:46.797867 (+ 296us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:56:46.797870 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:56:46.797872 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:56:46.797973 (+ 101us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:56:46.798398 (+ 425us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 02:56:46.798401 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 02:56:46.798492 (+ 91us) client_negotiation.cc:769] Sending connection context
0811 02:56:46.798594 (+ 102us) client_negotiation.cc:240] Negotiation successful
0811 02:56:46.798740 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":510}
I20260811 02:56:46.799768 6883 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.12.138.252" port: 44637 } has no permanent_uuid. Determining permanent_uuid...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.138.252 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786416707705494-12843-0/krb5kdc/kudu_127.12.138.252.keytab.
Entry for principal kudu/127.12.138.252 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786416707705494-12843-0/krb5kdc/kudu_127.12.138.252.keytab.
I20260811 02:56:46.800511 6888 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:56:46.800243 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.138.252:44637 (local address 127.0.0.1:50294)
0811 02:56:46.800364 (+ 121us) negotiation.cc:107] Waiting for socket to connect
0811 02:56:46.800447 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.138.252:44637: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
W20260811 02:56:46.800792 6827 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.12.138.252:44637: connect: Connection refused (error 111)
I20260811 02:56:46.801432 12843 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDawEwS/build/debug/bin/kudu
/tmp/dist-test-taskDawEwS/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786416707705494-12843-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786416707705494-12843-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786416707705494-12843-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786416707705494-12843-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.138.252:44637
--webserver_interface=127.12.138.252
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786416707705494-12843-0/krb5kdc/kudu_127.12.138.252.keytab
--principal=kudu/127.12.138.252
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.138.212:41187
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.12.138.254:44727,127.12.138.253:43519,127.12.138.252:44637
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786416707705494-12843-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786416707705494-12843-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786416707705494-12843-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 02:56:46.802325 6888 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:56:46.802076 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.138.252:44637 (local address 127.0.0.1:50296)
0811 02:56:46.802213 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0811 02:56:46.802267 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.138.252:44637: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":65}
W20260811 02:56:46.802585 6883 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.12.138.252:44637: Network error: Client connection negotiation failed: client connection to 127.12.138.252:44637: connect: Connection refused (error 111)
I20260811 02:56:46.822892 6797 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.12.138.252" port: 44637 } attempt: 1
I20260811 02:56:46.823805 6800 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:56:46.823504 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.138.252:44637 (local address 127.0.0.1:50310)
0811 02:56:46.823656 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0811 02:56:46.823728 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.138.252:44637: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
I20260811 02:56:46.824648 6800 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:56:46.824399 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.138.252:44637 (local address 127.0.0.1:50322)
0811 02:56:46.824540 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0811 02:56:46.824590 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.138.252:44637: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":60}
W20260811 02:56:46.824887 6797 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.12.138.252:44637: Network error: Client connection negotiation failed: client connection to 127.12.138.252:44637: connect: Connection refused (error 111)
I20260811 02:56:46.851960 6883 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.12.138.252" port: 44637 } attempt: 1
I20260811 02:56:46.853041 6888 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:56:46.852694 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.138.252:44637 (local address 127.0.0.1:50326)
0811 02:56:46.852874 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0811 02:56:46.852947 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.138.252:44637: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":71}
I20260811 02:56:46.853915 6888 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:56:46.853642 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.138.252:44637 (local address 127.0.0.1:50342)
0811 02:56:46.853780 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0811 02:56:46.853839 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.138.252:44637: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":60}
W20260811 02:56:46.854103 6883 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.12.138.252:44637: Network error: Client connection negotiation failed: client connection to 127.12.138.252:44637: connect: Connection refused (error 111)
I20260811 02:56:46.900086 6797 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.12.138.252" port: 44637 } attempt: 2
I20260811 02:56:46.901095 6800 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:56:46.900744 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.138.252:44637 (local address 127.0.0.1:50344)
0811 02:56:46.900931 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0811 02:56:46.901011 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.138.252:44637: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":79}
I20260811 02:56:46.901965 6800 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:56:46.901752 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.138.252:44637 (local address 127.0.0.1:50346)
0811 02:56:46.901859 (+ 107us) negotiation.cc:107] Waiting for socket to connect
0811 02:56:46.901907 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.138.252:44637: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
W20260811 02:56:46.902127 6797 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.12.138.252:44637: Network error: Client connection negotiation failed: client connection to 127.12.138.252:44637: connect: Connection refused (error 111)
W20260811 02:56:46.920413 6891 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260811 02:56:46.920687 6891 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 02:56:46.920719 6891 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 02:56:46.920758 6891 flags.cc:425] Enabled unsafe flag: --never_fsync=true
I20260811 02:56:46.922276 6883 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.12.138.252" port: 44637 } attempt: 2
I20260811 02:56:46.923346 6888 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:56:46.922882 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.138.252:44637 (local address 127.0.0.1:50360)
0811 02:56:46.923122 (+ 240us) negotiation.cc:107] Waiting for socket to connect
0811 02:56:46.923220 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.138.252:44637: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
I20260811 02:56:46.924490 6888 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:56:46.924203 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.138.252:44637 (local address 127.0.0.1:50368)
0811 02:56:46.924351 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0811 02:56:46.924406 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.138.252:44637: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":28}
W20260811 02:56:46.925199 6883 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.12.138.252:44637: Network error: Client connection negotiation failed: client connection to 127.12.138.252:44637: connect: Connection refused (error 111)
W20260811 02:56:46.927268 6891 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 02:56:46.927351 6891 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 02:56:46.927397 6891 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 02:56:46.927424 6891 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 02:56:46.927465 6891 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 02:56:46.934048 6891 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.138.212:41187
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786416707705494-12843-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786416707705494-12843-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.12.138.254:44727,127.12.138.253:43519,127.12.138.252:44637
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786416707705494-12843-0/krb5kdc/kudu_127.12.138.252.keytab
--principal=kudu/127.12.138.252
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.138.252:44637
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786416707705494-12843-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.12.138.252
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6891
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786416707705494-12843-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 02:09:49 UTC on ccycloud.quasar-fsnqqc.root.comops.site
build id 4579570
I20260811 02:56:46.935168 6891 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 02:56:46.936281 6891 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 02:56:46.945127 6898 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:56:46.945091 6899 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:56:46.945091 6902 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:56:46.945587 6891 server_base.cc:1034] running on GCE node
I20260811 02:56:46.946151 6891 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 02:56:46.947264 6891 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 02:56:46.948563 6891 hybrid_clock.cc:648] HybridClock initialized: now 1786417006948519 us; error 63 us; skew 500 ppm
Aug 11 02:56:46 dist-test-slave-9gcw krb5kdc[6698](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417006, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.138.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 02:56:46.951540 6891 init.cc:377] Logged in from keytab as kudu/127.12.138.252@KRBTEST.COM (short username kudu)
I20260811 02:56:46.952387 6891 webserver.cc:460] Webserver started at http://127.12.138.252:42629/ using document root <none> and password file <none>
I20260811 02:56:46.952905 6891 fs_manager.cc:359] Metadata directory not provided
I20260811 02:56:46.952966 6891 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 02:56:46.953135 6891 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 02:56:46.955201 6891 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786416707705494-12843-0/minicluster-data/master-2/data/instance:
uuid: "3c53c36ddfd84d68846da9a01be74fc2"
format_stamp: "Formatted at 2026-08-11 02:56:46 on dist-test-slave-9gcw"
server_key: "0913f9242c0494ef869d5f4b48a7f7d4"
server_key_iv: "30bbe9af53585fd98f87fefa46228fbd"
server_key_version: "encryptionkey@0"
I20260811 02:56:46.955765 6891 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786416707705494-12843-0/minicluster-data/master-2/wal/instance:
uuid: "3c53c36ddfd84d68846da9a01be74fc2"
format_stamp: "Formatted at 2026-08-11 02:56:46 on dist-test-slave-9gcw"
server_key: "0913f9242c0494ef869d5f4b48a7f7d4"
server_key_iv: "30bbe9af53585fd98f87fefa46228fbd"
server_key_version: "encryptionkey@0"
I20260811 02:56:46.959558 6891 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.004s
I20260811 02:56:46.962205 6911 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 02:56:46.963594 6891 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 02:56:46.963774 6891 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786416707705494-12843-0/minicluster-data/master-2/data,/tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786416707705494-12843-0/minicluster-data/master-2/wal
uuid: "3c53c36ddfd84d68846da9a01be74fc2"
format_stamp: "Formatted at 2026-08-11 02:56:46 on dist-test-slave-9gcw"
server_key: "0913f9242c0494ef869d5f4b48a7f7d4"
server_key_iv: "30bbe9af53585fd98f87fefa46228fbd"
server_key_version: "encryptionkey@0"
I20260811 02:56:46.963894 6891 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786416707705494-12843-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786416707705494-12843-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786416707705494-12843-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 02:56:46.990532 6891 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 02:56:46.993738 6891 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 02:56:46.993968 6891 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 02:56:47.001349 6797 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.12.138.252" port: 44637 } attempt: 3
I20260811 02:56:47.001780 6891 rpc_server.cc:307] RPC server started. Bound to: 127.12.138.252:44637
I20260811 02:56:47.002081 6973 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.138.252:44637 every 8 connection(s)
I20260811 02:56:47.003108 6891 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786416707705494-12843-0/minicluster-data/master-2/data/info.pb
I20260811 02:56:47.006258 6976 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 02:56:47.009953 12843 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDawEwS/build/debug/bin/kudu as pid 6891
I20260811 02:56:47.010111 12843 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDawEwS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786416707705494-12843-0/minicluster-data/master-2/wal/instance
I20260811 02:56:47.010425 12843 external_mini_cluster.cc:1428] Setting key 2339d30e062ebec5acb77561628dddfe
Aug 11 02:56:47 dist-test-slave-9gcw krb5kdc[6698](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417006, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.138.254@KRBTEST.COM for kudu/127.12.138.252@KRBTEST.COM
I20260811 02:56:47.012442 6976 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.12.138.254" port: 44727 } has no permanent_uuid. Determining permanent_uuid...
I20260811 02:56:47.016407 6883 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.12.138.252" port: 44637 } attempt: 3
I20260811 02:56:47.018065 6800 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:56:47.002070 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.138.252:44637 (local address 127.0.0.1:50376)
0811 02:56:47.002233 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0811 02:56:47.002254 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0811 02:56:47.002369 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:56:47.005288 (+ 2919us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:56:47.005292 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:56:47.005508 (+ 216us) client_negotiation.cc:189] Negotiated authn=SASL
0811 02:56:47.005638 (+ 130us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:56:47.005651 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:56:47.007267 (+ 1616us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:56:47.007271 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:56:47.008054 (+ 783us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:56:47.008068 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:56:47.009088 (+ 1020us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:56:47.009092 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:56:47.009236 (+ 144us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:56:47.009885 (+ 649us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 02:56:47.009909 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0811 02:56:47.011831 (+ 1922us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 02:56:47.014686 (+ 2855us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:56:47.014691 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:56:47.014694 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:56:47.014994 (+ 300us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:56:47.015466 (+ 472us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:56:47.015471 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:56:47.015473 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:56:47.015590 (+ 117us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:56:47.016525 (+ 935us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 02:56:47.016599 (+ 74us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 02:56:47.016719 (+ 120us) client_negotiation.cc:769] Sending connection context
0811 02:56:47.016847 (+ 128us) client_negotiation.cc:240] Negotiation successful
0811 02:56:47.017897 (+ 1050us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":67,"mutex_wait_us":3}
Aug 11 02:56:47 dist-test-slave-9gcw krb5kdc[6698](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417006, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.12.138.252@KRBTEST.COM
I20260811 02:56:47.018913 6974 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:56:47.003063 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50376 (local address 127.12.138.252:44637)
0811 02:56:47.003573 (+ 510us) server_negotiation.cc:206] Beginning negotiation
0811 02:56:47.003594 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0811 02:56:47.003685 (+ 91us) server_negotiation.cc:407] Connection header received
0811 02:56:47.004511 (+ 826us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:56:47.004530 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:56:47.004922 (+ 392us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:56:47.005334 (+ 412us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:56:47.005779 (+ 445us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:56:47.007097 (+ 1318us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:56:47.008242 (+ 1145us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:56:47.008942 (+ 700us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:56:47.009183 (+ 241us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:56:47.012097 (+ 2914us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:56:47.012104 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:56:47.012121 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:56:47.012168 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:56:47.014548 (+ 2380us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:56:47.015136 (+ 588us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:56:47.015142 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:56:47.015148 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:56:47.015314 (+ 166us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:56:47.015717 (+ 403us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:56:47.015720 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:56:47.015722 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:56:47.016326 (+ 604us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:56:47.016470 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0811 02:56:47.017596 (+ 1126us) server_negotiation.cc:299] Negotiation successful
0811 02:56:47.017911 (+ 315us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":356,"thread_start_us":149,"threads_started":1}
I20260811 02:56:47.023444 6979 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:56:47.011090 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50390 (local address 127.12.138.252:44637)
0811 02:56:47.011558 (+ 468us) server_negotiation.cc:206] Beginning negotiation
0811 02:56:47.011567 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 02:56:47.011892 (+ 325us) server_negotiation.cc:407] Connection header received
0811 02:56:47.012070 (+ 178us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:56:47.012073 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:56:47.012129 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:56:47.012226 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:56:47.013208 (+ 982us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:56:47.013849 (+ 641us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:56:47.014970 (+ 1121us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:56:47.015293 (+ 323us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:56:47.015406 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:56:47.018813 (+ 3407us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:56:47.018817 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:56:47.018820 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:56:47.018845 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:56:47.020790 (+ 1945us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:56:47.021358 (+ 568us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:56:47.021362 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:56:47.021365 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:56:47.021460 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:56:47.021856 (+ 396us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:56:47.021860 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:56:47.021862 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:56:47.022116 (+ 254us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:56:47.022334 (+ 218us) server_negotiation.cc:1035] Waiting for connection context
0811 02:56:47.023083 (+ 749us) server_negotiation.cc:299] Negotiation successful
0811 02:56:47.023212 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":405,"thread_start_us":158,"threads_started":1}
Aug 11 02:56:47 dist-test-slave-9gcw krb5kdc[6698](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417006, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.138.252@KRBTEST.COM for kudu/127.12.138.254@KRBTEST.COM
Aug 11 02:56:47 dist-test-slave-9gcw krb5kdc[6698](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417006, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.138.253@KRBTEST.COM for kudu/127.12.138.252@KRBTEST.COM
I20260811 02:56:47.025537 6797 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 71018b8f50c84cf4991af4f7257992a4: Bootstrap starting.
I20260811 02:56:47.028864 6888 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:56:47.016876 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.138.252:44637 (local address 127.0.0.1:50398)
0811 02:56:47.017219 (+ 343us) negotiation.cc:107] Waiting for socket to connect
0811 02:56:47.017238 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0811 02:56:47.017336 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:56:47.017842 (+ 506us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:56:47.017847 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:56:47.018042 (+ 195us) client_negotiation.cc:189] Negotiated authn=SASL
0811 02:56:47.018190 (+ 148us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:56:47.018200 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:56:47.020440 (+ 2240us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:56:47.020444 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:56:47.021325 (+ 881us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:56:47.021340 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:56:47.022119 (+ 779us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:56:47.022123 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:56:47.022225 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:56:47.022776 (+ 551us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 02:56:47.022799 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0811 02:56:47.024482 (+ 1683us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 02:56:47.027049 (+ 2567us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:56:47.027053 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:56:47.027057 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:56:47.027348 (+ 291us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:56:47.027696 (+ 348us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:56:47.027699 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:56:47.027701 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:56:47.027791 (+ 90us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:56:47.028306 (+ 515us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 02:56:47.028310 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 02:56:47.028428 (+ 118us) client_negotiation.cc:769] Sending connection context
0811 02:56:47.028523 (+ 95us) client_negotiation.cc:240] Negotiation successful
0811 02:56:47.028686 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":227}
I20260811 02:56:47.029095 6980 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:56:47.014729 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.138.254:44727 (local address 127.0.0.1:35318)
0811 02:56:47.015184 (+ 455us) negotiation.cc:107] Waiting for socket to connect
0811 02:56:47.015246 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0811 02:56:47.015377 (+ 131us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:56:47.015954 (+ 577us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:56:47.015961 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:56:47.016532 (+ 571us) client_negotiation.cc:189] Negotiated authn=SASL
0811 02:56:47.016721 (+ 189us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:56:47.016735 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:56:47.018587 (+ 1852us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:56:47.018590 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:56:47.020249 (+ 1659us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:56:47.020258 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:56:47.020910 (+ 652us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:56:47.020913 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:56:47.021041 (+ 128us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:56:47.022260 (+ 1219us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 02:56:47.022274 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0811 02:56:47.024229 (+ 1955us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 02:56:47.026931 (+ 2702us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:56:47.026943 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:56:47.026954 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:56:47.027276 (+ 322us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:56:47.027672 (+ 396us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:56:47.027690 (+ 18us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:56:47.027691 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:56:47.027786 (+ 95us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:56:47.028374 (+ 588us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 02:56:47.028381 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 02:56:47.028502 (+ 121us) client_negotiation.cc:769] Sending connection context
0811 02:56:47.028674 (+ 172us) client_negotiation.cc:240] Negotiation successful
0811 02:56:47.028871 (+ 197us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":227,"thread_start_us":110,"threads_started":1}
I20260811 02:56:47.029440 6797 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 71018b8f50c84cf4991af4f7257992a4: Neither blocks nor log segments found. Creating new log.
I20260811 02:56:47.029887 6801 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:56:47.015581 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35318 (local address 127.12.138.254:44727)
0811 02:56:47.015724 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0811 02:56:47.015728 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 02:56:47.015740 (+ 12us) server_negotiation.cc:407] Connection header received
0811 02:56:47.015781 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:56:47.015783 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:56:47.015818 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:56:47.015917 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:56:47.016882 (+ 965us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:56:47.017694 (+ 812us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:56:47.020377 (+ 2683us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:56:47.020701 (+ 324us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:56:47.020798 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:56:47.024388 (+ 3590us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:56:47.024392 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:56:47.024395 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:56:47.024423 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:56:47.026772 (+ 2349us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:56:47.027420 (+ 648us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:56:47.027426 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:56:47.027432 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:56:47.027549 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:56:47.027912 (+ 363us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:56:47.027915 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:56:47.027917 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:56:47.028202 (+ 285us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:56:47.028305 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0811 02:56:47.029552 (+ 1247us) server_negotiation.cc:299] Negotiation successful
0811 02:56:47.029716 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":22,"server-negotiator.queue_time_us":60}
I20260811 02:56:47.030313 6981 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:56:47.017181 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50398 (local address 127.12.138.252:44637)
0811 02:56:47.017565 (+ 384us) server_negotiation.cc:206] Beginning negotiation
0811 02:56:47.017574 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 02:56:47.017588 (+ 14us) server_negotiation.cc:407] Connection header received
0811 02:56:47.017623 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:56:47.017626 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:56:47.017667 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:56:47.018319 (+ 652us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:56:47.018465 (+ 146us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:56:47.019679 (+ 1214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:56:47.021564 (+ 1885us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:56:47.021955 (+ 391us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:56:47.022441 (+ 486us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:56:47.024929 (+ 2488us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:56:47.024936 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:56:47.024944 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:56:47.024977 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:56:47.026913 (+ 1936us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:56:47.027481 (+ 568us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:56:47.027488 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:56:47.027491 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:56:47.027573 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:56:47.027916 (+ 343us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:56:47.027919 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:56:47.027921 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:56:47.028159 (+ 238us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:56:47.029770 (+ 1611us) server_negotiation.cc:1035] Waiting for connection context
0811 02:56:47.029849 (+ 79us) server_negotiation.cc:299] Negotiation successful
0811 02:56:47.030120 (+ 271us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":305,"thread_start_us":116,"threads_started":1}
I20260811 02:56:47.031113 6976 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.12.138.253" port: 43519 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 02:56:47 dist-test-slave-9gcw krb5kdc[6698](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417006, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.12.138.254@KRBTEST.COM
I20260811 02:56:47.031164 6797 log.cc:824] T 00000000000000000000000000000000 P 71018b8f50c84cf4991af4f7257992a4: Log is configured to *not* fsync() on all Append() calls
I20260811 02:56:47.034345 6883 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9cc09b2071ea409587a0ceb8a1d7c94a: Bootstrap starting.
I20260811 02:56:47.036952 6797 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 71018b8f50c84cf4991af4f7257992a4: No bootstrap required, opened a new log
Aug 11 02:56:47 dist-test-slave-9gcw krb5kdc[6698](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417006, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.138.252@KRBTEST.COM for kudu/127.12.138.253@KRBTEST.COM
I20260811 02:56:47.037623 6883 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9cc09b2071ea409587a0ceb8a1d7c94a: Neither blocks nor log segments found. Creating new log.
I20260811 02:56:47.038314 6982 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:56:47.025075 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35320 (local address 127.12.138.254:44727)
0811 02:56:47.025463 (+ 388us) server_negotiation.cc:206] Beginning negotiation
0811 02:56:47.025477 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0811 02:56:47.025496 (+ 19us) server_negotiation.cc:407] Connection header received
0811 02:56:47.025538 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:56:47.025544 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:56:47.025603 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:56:47.025793 (+ 190us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:56:47.026690 (+ 897us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:56:47.027534 (+ 844us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:56:47.028705 (+ 1171us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:56:47.029089 (+ 384us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:56:47.029297 (+ 208us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:56:47.032652 (+ 3355us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:56:47.032657 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:56:47.032660 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:56:47.032697 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:56:47.035063 (+ 2366us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:56:47.035646 (+ 583us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:56:47.035650 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:56:47.035652 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:56:47.035751 (+ 99us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:56:47.036404 (+ 653us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:56:47.036408 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:56:47.036410 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:56:47.036681 (+ 271us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:56:47.036857 (+ 176us) server_negotiation.cc:1035] Waiting for connection context
0811 02:56:47.037961 (+ 1104us) server_negotiation.cc:299] Negotiation successful
0811 02:56:47.038100 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":284,"thread_start_us":148,"threads_started":1}
I20260811 02:56:47.038631 6883 log.cc:824] T 00000000000000000000000000000000 P 9cc09b2071ea409587a0ceb8a1d7c94a: Log is configured to *not* fsync() on all Append() calls
I20260811 02:56:47.041985 6797 raft_consensus.cc:348] T 00000000000000000000000000000000 P 71018b8f50c84cf4991af4f7257992a4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71018b8f50c84cf4991af4f7257992a4" member_type: VOTER last_known_addr { host: "127.12.138.254" port: 44727 } } peers { permanent_uuid: "9cc09b2071ea409587a0ceb8a1d7c94a" member_type: VOTER last_known_addr { host: "127.12.138.253" port: 43519 } } peers { permanent_uuid: "3c53c36ddfd84d68846da9a01be74fc2" member_type: VOTER last_known_addr { host: "127.12.138.252" port: 44637 } }
I20260811 02:56:47.042263 6797 raft_consensus.cc:729] T 00000000000000000000000000000000 P 71018b8f50c84cf4991af4f7257992a4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 71018b8f50c84cf4991af4f7257992a4, State: Initialized, Role: FOLLOWER
I20260811 02:56:47.042371 6980 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:56:47.031686 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.138.253:43519 (local address 127.0.0.1:56560)
0811 02:56:47.031982 (+ 296us) negotiation.cc:107] Waiting for socket to connect
0811 02:56:47.032016 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0811 02:56:47.032106 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:56:47.032415 (+ 309us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:56:47.032418 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:56:47.032629 (+ 211us) client_negotiation.cc:189] Negotiated authn=SASL
0811 02:56:47.032760 (+ 131us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:56:47.032770 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:56:47.033902 (+ 1132us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:56:47.033905 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:56:47.034768 (+ 863us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:56:47.034780 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:56:47.035537 (+ 757us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:56:47.035541 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:56:47.035682 (+ 141us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:56:47.036165 (+ 483us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 02:56:47.036182 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 02:56:47.037975 (+ 1793us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 02:56:47.040374 (+ 2399us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:56:47.040377 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:56:47.040379 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:56:47.040680 (+ 301us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:56:47.041026 (+ 346us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:56:47.041033 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:56:47.041039 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:56:47.041181 (+ 142us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:56:47.041773 (+ 592us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 02:56:47.041776 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 02:56:47.041882 (+ 106us) client_negotiation.cc:769] Sending connection context
0811 02:56:47.041995 (+ 113us) client_negotiation.cc:240] Negotiation successful
0811 02:56:47.042166 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":235}
I20260811 02:56:47.042629 6797 consensus_queue.cc:260] T 00000000000000000000000000000000 P 71018b8f50c84cf4991af4f7257992a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71018b8f50c84cf4991af4f7257992a4" member_type: VOTER last_known_addr { host: "127.12.138.254" port: 44727 } } peers { permanent_uuid: "9cc09b2071ea409587a0ceb8a1d7c94a" member_type: VOTER last_known_addr { host: "127.12.138.253" port: 43519 } } peers { permanent_uuid: "3c53c36ddfd84d68846da9a01be74fc2" member_type: VOTER last_known_addr { host: "127.12.138.252" port: 44637 } }
I20260811 02:56:47.043017 6883 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9cc09b2071ea409587a0ceb8a1d7c94a: No bootstrap required, opened a new log
I20260811 02:56:47.043251 6986 sys_catalog.cc:455] T 00000000000000000000000000000000 P 71018b8f50c84cf4991af4f7257992a4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71018b8f50c84cf4991af4f7257992a4" member_type: VOTER last_known_addr { host: "127.12.138.254" port: 44727 } } peers { permanent_uuid: "9cc09b2071ea409587a0ceb8a1d7c94a" member_type: VOTER last_known_addr { host: "127.12.138.253" port: 43519 } } peers { permanent_uuid: "3c53c36ddfd84d68846da9a01be74fc2" member_type: VOTER last_known_addr { host: "127.12.138.252" port: 44637 } } }
I20260811 02:56:47.043398 6986 sys_catalog.cc:458] T 00000000000000000000000000000000 P 71018b8f50c84cf4991af4f7257992a4 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 02:56:47.044248 6797 sys_catalog.cc:564] T 00000000000000000000000000000000 P 71018b8f50c84cf4991af4f7257992a4 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 02:56:47.043001 6887 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:56:47.031781 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56560 (local address 127.12.138.253:43519)
0811 02:56:47.031907 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0811 02:56:47.031911 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 02:56:47.032192 (+ 281us) server_negotiation.cc:407] Connection header received
0811 02:56:47.032227 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:56:47.032229 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:56:47.032268 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:56:47.032364 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:56:47.032917 (+ 553us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:56:47.033747 (+ 830us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:56:47.034934 (+ 1187us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:56:47.035282 (+ 348us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:56:47.035410 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:56:47.038273 (+ 2863us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:56:47.038277 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:56:47.038280 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:56:47.038307 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:56:47.040222 (+ 1915us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:56:47.040812 (+ 590us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:56:47.040829 (+ 17us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:56:47.040834 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:56:47.040933 (+ 99us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:56:47.041314 (+ 381us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:56:47.041321 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:56:47.041327 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:56:47.041611 (+ 284us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:56:47.041719 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0811 02:56:47.042717 (+ 998us) server_negotiation.cc:299] Negotiation successful
0811 02:56:47.042848 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
Aug 11 02:56:47 dist-test-slave-9gcw krb5kdc[6698](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417006, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.12.138.253@KRBTEST.COM
I20260811 02:56:47.048424 6883 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9cc09b2071ea409587a0ceb8a1d7c94a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71018b8f50c84cf4991af4f7257992a4" member_type: VOTER last_known_addr { host: "127.12.138.254" port: 44727 } } peers { permanent_uuid: "9cc09b2071ea409587a0ceb8a1d7c94a" member_type: VOTER last_known_addr { host: "127.12.138.253" port: 43519 } } peers { permanent_uuid: "3c53c36ddfd84d68846da9a01be74fc2" member_type: VOTER last_known_addr { host: "127.12.138.252" port: 44637 } }
I20260811 02:56:47.048756 6976 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.12.138.252" port: 44637 } has no permanent_uuid. Determining permanent_uuid...
I20260811 02:56:47.048797 6883 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9cc09b2071ea409587a0ceb8a1d7c94a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9cc09b2071ea409587a0ceb8a1d7c94a, State: Initialized, Role: FOLLOWER
I20260811 02:56:47.049381 6883 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9cc09b2071ea409587a0ceb8a1d7c94a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71018b8f50c84cf4991af4f7257992a4" member_type: VOTER last_known_addr { host: "127.12.138.254" port: 44727 } } peers { permanent_uuid: "9cc09b2071ea409587a0ceb8a1d7c94a" member_type: VOTER last_known_addr { host: "127.12.138.253" port: 43519 } } peers { permanent_uuid: "3c53c36ddfd84d68846da9a01be74fc2" member_type: VOTER last_known_addr { host: "127.12.138.252" port: 44637 } }
I20260811 02:56:47.052103 6883 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9cc09b2071ea409587a0ceb8a1d7c94a [sys.catalog]: configured and running, proceeding with master startup.
I20260811 02:56:47.052907 6985 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:56:47.040198 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56568 (local address 127.12.138.253:43519)
0811 02:56:47.040594 (+ 396us) server_negotiation.cc:206] Beginning negotiation
0811 02:56:47.040603 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 02:56:47.040621 (+ 18us) server_negotiation.cc:407] Connection header received
0811 02:56:47.040661 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:56:47.040666 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:56:47.040718 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:56:47.040855 (+ 137us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:56:47.041776 (+ 921us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:56:47.042732 (+ 956us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:56:47.043965 (+ 1233us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:56:47.044351 (+ 386us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:56:47.044464 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:56:47.048168 (+ 3704us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:56:47.048172 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:56:47.048188 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:56:47.048217 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:56:47.050325 (+ 2108us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:56:47.050873 (+ 548us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:56:47.050877 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:56:47.050879 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:56:47.050976 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:56:47.051304 (+ 328us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:56:47.051307 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:56:47.051309 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:56:47.051557 (+ 248us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:56:47.051772 (+ 215us) server_negotiation.cc:1035] Waiting for connection context
0811 02:56:47.052497 (+ 725us) server_negotiation.cc:299] Negotiation successful
0811 02:56:47.052683 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":303,"thread_start_us":139,"threads_started":1}
Aug 11 02:56:47 dist-test-slave-9gcw krb5kdc[6698](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417006, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.138.252@KRBTEST.COM for kudu/127.12.138.252@KRBTEST.COM
I20260811 02:56:47.051262 6713 master_runner.cc:186] Error getting master registration for 127.12.138.253:43519: OK, instance_id { permanent_uuid: "9cc09b2071ea409587a0ceb8a1d7c94a" instance_seqno: 1786417006760048 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260811 02:56:47.058125 12843 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 02:56:47.058234 6713 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260811 02:56:47.059238 7000 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 71018b8f50c84cf4991af4f7257992a4: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 02:56:47.059324 7000 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 02:56:47.059623 7002 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9cc09b2071ea409587a0ceb8a1d7c94a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71018b8f50c84cf4991af4f7257992a4" member_type: VOTER last_known_addr { host: "127.12.138.254" port: 44727 } } peers { permanent_uuid: "9cc09b2071ea409587a0ceb8a1d7c94a" member_type: VOTER last_known_addr { host: "127.12.138.253" port: 43519 } } peers { permanent_uuid: "3c53c36ddfd84d68846da9a01be74fc2" member_type: VOTER last_known_addr { host: "127.12.138.252" port: 44637 } } }
I20260811 02:56:47.059795 7002 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9cc09b2071ea409587a0ceb8a1d7c94a [sys.catalog]: This master's current role is: FOLLOWER
I20260811 02:56:47.059720 6764 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "71018b8f50c84cf4991af4f7257992a4"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35320
I20260811 02:56:47.059949 6764 raft_consensus.cc:482] T 00000000000000000000000000000000 P 71018b8f50c84cf4991af4f7257992a4 [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260811 02:56:47.060057 6980 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:56:47.049330 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.138.252:44637 (local address 127.0.0.1:50412)
0811 02:56:47.049646 (+ 316us) negotiation.cc:107] Waiting for socket to connect
0811 02:56:47.049662 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 02:56:47.049777 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:56:47.050126 (+ 349us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:56:47.050129 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:56:47.050322 (+ 193us) client_negotiation.cc:189] Negotiated authn=SASL
0811 02:56:47.050420 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:56:47.050428 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:56:47.051547 (+ 1119us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:56:47.051551 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:56:47.052427 (+ 876us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:56:47.052439 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:56:47.053378 (+ 939us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:56:47.053391 (+ 13us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:56:47.053483 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:56:47.054026 (+ 543us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 02:56:47.054039 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0811 02:56:47.055838 (+ 1799us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 02:56:47.058214 (+ 2376us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:56:47.058217 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:56:47.058219 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:56:47.058456 (+ 237us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:56:47.058750 (+ 294us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:56:47.058753 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:56:47.058755 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:56:47.058836 (+ 81us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:56:47.059304 (+ 468us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 02:56:47.059306 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 02:56:47.059402 (+ 96us) client_negotiation.cc:769] Sending connection context
0811 02:56:47.059575 (+ 173us) client_negotiation.cc:240] Negotiation successful
0811 02:56:47.059810 (+ 235us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":209}
I20260811 02:56:47.060070 6764 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 71018b8f50c84cf4991af4f7257992a4 [term 0 FOLLOWER]: Advancing to term 1
I20260811 02:56:47.060528 6981 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:56:47.049624 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50412 (local address 127.12.138.252:44637)
0811 02:56:47.049762 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0811 02:56:47.049767 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 02:56:47.049783 (+ 16us) server_negotiation.cc:407] Connection header received
0811 02:56:47.049877 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:56:47.049880 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:56:47.049927 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:56:47.050013 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:56:47.050544 (+ 531us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:56:47.051425 (+ 881us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:56:47.052555 (+ 1130us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:56:47.053107 (+ 552us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:56:47.053267 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:56:47.056013 (+ 2746us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:56:47.056020 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:56:47.056024 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:56:47.056051 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:56:47.058096 (+ 2045us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:56:47.058558 (+ 462us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:56:47.058561 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:56:47.058563 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:56:47.058655 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:56:47.058934 (+ 279us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:56:47.058939 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:56:47.058940 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:56:47.059167 (+ 227us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:56:47.059271 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0811 02:56:47.060268 (+ 997us) server_negotiation.cc:299] Negotiation successful
0811 02:56:47.060397 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260811 02:56:47.061560 6764 raft_consensus.cc:504] T 00000000000000000000000000000000 P 71018b8f50c84cf4991af4f7257992a4 [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71018b8f50c84cf4991af4f7257992a4" member_type: VOTER last_known_addr { host: "127.12.138.254" port: 44727 } } peers { permanent_uuid: "9cc09b2071ea409587a0ceb8a1d7c94a" member_type: VOTER last_known_addr { host: "127.12.138.253" port: 43519 } } peers { permanent_uuid: "3c53c36ddfd84d68846da9a01be74fc2" member_type: VOTER last_known_addr { host: "127.12.138.252" port: 44637 } }
I20260811 02:56:47.062602 6764 leader_election.cc:288] T 00000000000000000000000000000000 P 71018b8f50c84cf4991af4f7257992a4 [CANDIDATE]: Term 1 election: Requested vote from peers 9cc09b2071ea409587a0ceb8a1d7c94a (127.12.138.253:43519), 3c53c36ddfd84d68846da9a01be74fc2 (127.12.138.252:44637)
I20260811 02:56:47.063289 6852 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "71018b8f50c84cf4991af4f7257992a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "9cc09b2071ea409587a0ceb8a1d7c94a"
I20260811 02:56:47.063542 6852 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9cc09b2071ea409587a0ceb8a1d7c94a [term 0 FOLLOWER]: Advancing to term 1
I20260811 02:56:47.065137 6852 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 9cc09b2071ea409587a0ceb8a1d7c94a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 71018b8f50c84cf4991af4f7257992a4 in term 1.
I20260811 02:56:47.063344 6804 master_runner.cc:186] Error getting master registration for 127.12.138.252:44637: OK, instance_id { permanent_uuid: "3c53c36ddfd84d68846da9a01be74fc2" instance_seqno: 1786417006999538 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260811 02:56:47.067444 6736 leader_election.cc:302] T 00000000000000000000000000000000 P 71018b8f50c84cf4991af4f7257992a4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 71018b8f50c84cf4991af4f7257992a4, 9cc09b2071ea409587a0ceb8a1d7c94a; no voters:
I20260811 02:56:47.067853 6986 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 71018b8f50c84cf4991af4f7257992a4 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 02:56:47.068115 6986 raft_consensus.cc:686] T 00000000000000000000000000000000 P 71018b8f50c84cf4991af4f7257992a4 [term 1 LEADER]: Becoming Leader. State: Replica: 71018b8f50c84cf4991af4f7257992a4, State: Running, Role: LEADER
I20260811 02:56:47.068537 6986 consensus_queue.cc:237] T 00000000000000000000000000000000 P 71018b8f50c84cf4991af4f7257992a4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71018b8f50c84cf4991af4f7257992a4" member_type: VOTER last_known_addr { host: "127.12.138.254" port: 44727 } } peers { permanent_uuid: "9cc09b2071ea409587a0ceb8a1d7c94a" member_type: VOTER last_known_addr { host: "127.12.138.253" port: 43519 } } peers { permanent_uuid: "3c53c36ddfd84d68846da9a01be74fc2" member_type: VOTER last_known_addr { host: "127.12.138.252" port: 44637 } }
W20260811 02:56:47.069204 7017 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 9cc09b2071ea409587a0ceb8a1d7c94a: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 02:56:47.069278 7017 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 02:56:47.069257 6976 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3c53c36ddfd84d68846da9a01be74fc2: Bootstrap starting.
I20260811 02:56:47.070860 6986 sys_catalog.cc:455] T 00000000000000000000000000000000 P 71018b8f50c84cf4991af4f7257992a4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 71018b8f50c84cf4991af4f7257992a4. Latest consensus state: current_term: 1 leader_uuid: "71018b8f50c84cf4991af4f7257992a4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71018b8f50c84cf4991af4f7257992a4" member_type: VOTER last_known_addr { host: "127.12.138.254" port: 44727 } } peers { permanent_uuid: "9cc09b2071ea409587a0ceb8a1d7c94a" member_type: VOTER last_known_addr { host: "127.12.138.253" port: 43519 } } peers { permanent_uuid: "3c53c36ddfd84d68846da9a01be74fc2" member_type: VOTER last_known_addr { host: "127.12.138.252" port: 44637 } } }
I20260811 02:56:47.070982 6986 sys_catalog.cc:458] T 00000000000000000000000000000000 P 71018b8f50c84cf4991af4f7257992a4 [sys.catalog]: This master's current role is: LEADER
I20260811 02:56:47.071282 6804 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260811 02:56:47.071658 7021 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 02:56:47.072340 6976 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3c53c36ddfd84d68846da9a01be74fc2: Neither blocks nor log segments found. Creating new log.
I20260811 02:56:47.073529 7021 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 02:56:47.075888 6976 log.cc:824] T 00000000000000000000000000000000 P 3c53c36ddfd84d68846da9a01be74fc2: Log is configured to *not* fsync() on all Append() calls
I20260811 02:56:47.083372 6852 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 9cc09b2071ea409587a0ceb8a1d7c94a [term 1 FOLLOWER]: Refusing update from remote peer 71018b8f50c84cf4991af4f7257992a4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 02:56:47.083897 6976 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3c53c36ddfd84d68846da9a01be74fc2: No bootstrap required, opened a new log
I20260811 02:56:47.084190 6986 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 71018b8f50c84cf4991af4f7257992a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9cc09b2071ea409587a0ceb8a1d7c94a" member_type: VOTER last_known_addr { host: "127.12.138.253" port: 43519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 02:56:47.089452 7002 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9cc09b2071ea409587a0ceb8a1d7c94a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 71018b8f50c84cf4991af4f7257992a4. Latest consensus state: current_term: 1 leader_uuid: "71018b8f50c84cf4991af4f7257992a4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71018b8f50c84cf4991af4f7257992a4" member_type: VOTER last_known_addr { host: "127.12.138.254" port: 44727 } } peers { permanent_uuid: "9cc09b2071ea409587a0ceb8a1d7c94a" member_type: VOTER last_known_addr { host: "127.12.138.253" port: 43519 } } peers { permanent_uuid: "3c53c36ddfd84d68846da9a01be74fc2" member_type: VOTER last_known_addr { host: "127.12.138.252" port: 44637 } } }
I20260811 02:56:47.089644 7002 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9cc09b2071ea409587a0ceb8a1d7c94a [sys.catalog]: This master's current role is: FOLLOWER
I20260811 02:56:47.089795 6976 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3c53c36ddfd84d68846da9a01be74fc2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71018b8f50c84cf4991af4f7257992a4" member_type: VOTER last_known_addr { host: "127.12.138.254" port: 44727 } } peers { permanent_uuid: "9cc09b2071ea409587a0ceb8a1d7c94a" member_type: VOTER last_known_addr { host: "127.12.138.253" port: 43519 } } peers { permanent_uuid: "3c53c36ddfd84d68846da9a01be74fc2" member_type: VOTER last_known_addr { host: "127.12.138.252" port: 44637 } }
I20260811 02:56:47.090142 6976 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3c53c36ddfd84d68846da9a01be74fc2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3c53c36ddfd84d68846da9a01be74fc2, State: Initialized, Role: FOLLOWER
I20260811 02:56:47.090552 6976 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3c53c36ddfd84d68846da9a01be74fc2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71018b8f50c84cf4991af4f7257992a4" member_type: VOTER last_known_addr { host: "127.12.138.254" port: 44727 } } peers { permanent_uuid: "9cc09b2071ea409587a0ceb8a1d7c94a" member_type: VOTER last_known_addr { host: "127.12.138.253" port: 43519 } } peers { permanent_uuid: "3c53c36ddfd84d68846da9a01be74fc2" member_type: VOTER last_known_addr { host: "127.12.138.252" port: 44637 } }
I20260811 02:56:47.092928 6976 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3c53c36ddfd84d68846da9a01be74fc2 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 02:56:47.093000 6944 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "71018b8f50c84cf4991af4f7257992a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "3c53c36ddfd84d68846da9a01be74fc2"
I20260811 02:56:47.093219 6946 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3c53c36ddfd84d68846da9a01be74fc2 [term 0 FOLLOWER]: Advancing to term 1
I20260811 02:56:47.097393 7019 sys_catalog.cc:455] T 00000000000000000000000000000000 P 71018b8f50c84cf4991af4f7257992a4 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "71018b8f50c84cf4991af4f7257992a4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71018b8f50c84cf4991af4f7257992a4" member_type: VOTER last_known_addr { host: "127.12.138.254" port: 44727 } } peers { permanent_uuid: "9cc09b2071ea409587a0ceb8a1d7c94a" member_type: VOTER last_known_addr { host: "127.12.138.253" port: 43519 } } peers { permanent_uuid: "3c53c36ddfd84d68846da9a01be74fc2" member_type: VOTER last_known_addr { host: "127.12.138.252" port: 44637 } } }
I20260811 02:56:47.097563 7019 sys_catalog.cc:458] T 00000000000000000000000000000000 P 71018b8f50c84cf4991af4f7257992a4 [sys.catalog]: This master's current role is: LEADER
I20260811 02:56:47.098194 6946 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 3c53c36ddfd84d68846da9a01be74fc2 [term 1 FOLLOWER]: Refusing update from remote peer 71018b8f50c84cf4991af4f7257992a4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 02:56:47.098977 7002 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9cc09b2071ea409587a0ceb8a1d7c94a [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "71018b8f50c84cf4991af4f7257992a4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71018b8f50c84cf4991af4f7257992a4" member_type: VOTER last_known_addr { host: "127.12.138.254" port: 44727 } } peers { permanent_uuid: "9cc09b2071ea409587a0ceb8a1d7c94a" member_type: VOTER last_known_addr { host: "127.12.138.253" port: 43519 } } peers { permanent_uuid: "3c53c36ddfd84d68846da9a01be74fc2" member_type: VOTER last_known_addr { host: "127.12.138.252" port: 44637 } } }
I20260811 02:56:47.099107 7002 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9cc09b2071ea409587a0ceb8a1d7c94a [sys.catalog]: This master's current role is: FOLLOWER
I20260811 02:56:47.099694 6986 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 71018b8f50c84cf4991af4f7257992a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c53c36ddfd84d68846da9a01be74fc2" member_type: VOTER last_known_addr { host: "127.12.138.252" port: 44637 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 02:56:47.100163 6944 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 3c53c36ddfd84d68846da9a01be74fc2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 71018b8f50c84cf4991af4f7257992a4 in term 1.
I20260811 02:56:47.104766 7027 mvcc.cc:205] Tried to move back new op lower bound from 7317164061000220672 to 7317164060957630464. Current Snapshot: MvccSnapshot[applied={T|T < 7317164061000220672}]
I20260811 02:56:47.105283 7032 mvcc.cc:205] Tried to move back new op lower bound from 7317164061000220672 to 7317164060957630464. Current Snapshot: MvccSnapshot[applied={T|T < 7317164061000220672}]
I20260811 02:56:47.107388 7037 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3c53c36ddfd84d68846da9a01be74fc2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "71018b8f50c84cf4991af4f7257992a4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71018b8f50c84cf4991af4f7257992a4" member_type: VOTER last_known_addr { host: "127.12.138.254" port: 44727 } } peers { permanent_uuid: "9cc09b2071ea409587a0ceb8a1d7c94a" member_type: VOTER last_known_addr { host: "127.12.138.253" port: 43519 } } peers { permanent_uuid: "3c53c36ddfd84d68846da9a01be74fc2" member_type: VOTER last_known_addr { host: "127.12.138.252" port: 44637 } } }
I20260811 02:56:47.107556 7037 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3c53c36ddfd84d68846da9a01be74fc2 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 02:56:47.107467 7034 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3c53c36ddfd84d68846da9a01be74fc2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 71018b8f50c84cf4991af4f7257992a4. Latest consensus state: current_term: 1 leader_uuid: "71018b8f50c84cf4991af4f7257992a4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71018b8f50c84cf4991af4f7257992a4" member_type: VOTER last_known_addr { host: "127.12.138.254" port: 44727 } } peers { permanent_uuid: "9cc09b2071ea409587a0ceb8a1d7c94a" member_type: VOTER last_known_addr { host: "127.12.138.253" port: 43519 } } peers { permanent_uuid: "3c53c36ddfd84d68846da9a01be74fc2" member_type: VOTER last_known_addr { host: "127.12.138.252" port: 44637 } } }
I20260811 02:56:47.107825 7034 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3c53c36ddfd84d68846da9a01be74fc2 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 02:56:47.107685 7037 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3c53c36ddfd84d68846da9a01be74fc2 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "71018b8f50c84cf4991af4f7257992a4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71018b8f50c84cf4991af4f7257992a4" member_type: VOTER last_known_addr { host: "127.12.138.254" port: 44727 } } peers { permanent_uuid: "9cc09b2071ea409587a0ceb8a1d7c94a" member_type: VOTER last_known_addr { host: "127.12.138.253" port: 43519 } } peers { permanent_uuid: "3c53c36ddfd84d68846da9a01be74fc2" member_type: VOTER last_known_addr { host: "127.12.138.252" port: 44637 } } }
I20260811 02:56:47.108044 7037 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3c53c36ddfd84d68846da9a01be74fc2 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 02:56:47.108963 6985 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:56:47.077733 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56580 (local address 127.12.138.253:43519)
0811 02:56:47.077902 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0811 02:56:47.077910 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 02:56:47.090417 (+ 12507us) server_negotiation.cc:407] Connection header received
0811 02:56:47.091219 (+ 802us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:56:47.091222 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:56:47.091277 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:56:47.091362 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:56:47.092429 (+ 1067us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:56:47.093265 (+ 836us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:56:47.094528 (+ 1263us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:56:47.094991 (+ 463us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:56:47.095084 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:56:47.101298 (+ 6214us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:56:47.101304 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:56:47.101309 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:56:47.101350 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:56:47.103597 (+ 2247us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:56:47.104788 (+ 1191us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:56:47.104793 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:56:47.104795 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:56:47.104896 (+ 101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:56:47.105284 (+ 388us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:56:47.105290 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:56:47.105295 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:56:47.105563 (+ 268us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:56:47.105694 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0811 02:56:47.106179 (+ 485us) server_negotiation.cc:299] Negotiation successful
0811 02:56:47.108735 (+ 2556us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260811 02:56:47.112274 7046 mvcc.cc:205] Tried to move back new op lower bound from 7317164061000220672 to 7317164060957630464. Current Snapshot: MvccSnapshot[applied={T|T < 7317164061000220672}]
I20260811 02:56:47.112429 6982 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:56:47.076902 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35322 (local address 127.12.138.254:44727)
0811 02:56:47.077037 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0811 02:56:47.077042 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 02:56:47.081925 (+ 4883us) server_negotiation.cc:407] Connection header received
0811 02:56:47.082721 (+ 796us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:56:47.082725 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:56:47.082849 (+ 124us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:56:47.082959 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:56:47.086111 (+ 3152us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:56:47.087044 (+ 933us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:56:47.090526 (+ 3482us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:56:47.090954 (+ 428us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:56:47.091054 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:56:47.106719 (+ 15665us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:56:47.106725 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:56:47.106730 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:56:47.106787 (+ 57us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:56:47.110030 (+ 3243us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:56:47.110690 (+ 660us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:56:47.110694 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:56:47.110696 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:56:47.110801 (+ 105us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:56:47.111213 (+ 412us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:56:47.111217 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:56:47.111219 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:56:47.111496 (+ 277us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:56:47.111639 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0811 02:56:47.112063 (+ 424us) server_negotiation.cc:299] Negotiation successful
0811 02:56:47.112244 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260811 02:56:47.117920 7021 catalog_manager.cc:1347] Generated new cluster ID: 49ca592dcc1243b7814e881decd6bea6
I20260811 02:56:47.117997 7021 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 02:56:47.118299 6981 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:56:47.078103 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50428 (local address 127.12.138.252:44637)
0811 02:56:47.078226 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0811 02:56:47.078231 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 02:56:47.091043 (+ 12812us) server_negotiation.cc:407] Connection header received
0811 02:56:47.091239 (+ 196us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:56:47.091242 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:56:47.091292 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:56:47.091353 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:56:47.094086 (+ 2733us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:56:47.094961 (+ 875us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:56:47.103924 (+ 8963us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:56:47.104377 (+ 453us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:56:47.104479 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:56:47.114096 (+ 9617us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:56:47.114101 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:56:47.114104 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:56:47.114135 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:56:47.116092 (+ 1957us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:56:47.116627 (+ 535us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:56:47.116631 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:56:47.116633 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:56:47.116736 (+ 103us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:56:47.117083 (+ 347us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:56:47.117086 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:56:47.117089 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:56:47.117385 (+ 296us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:56:47.117559 (+ 174us) server_negotiation.cc:1035] Waiting for connection context
0811 02:56:47.117982 (+ 423us) server_negotiation.cc:299] Negotiation successful
0811 02:56:47.118117 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
W20260811 02:56:47.128762 7054 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 3c53c36ddfd84d68846da9a01be74fc2: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 02:56:47.131747 7054 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 02:56:47.133222 7019 sys_catalog.cc:455] T 00000000000000000000000000000000 P 71018b8f50c84cf4991af4f7257992a4 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "71018b8f50c84cf4991af4f7257992a4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71018b8f50c84cf4991af4f7257992a4" member_type: VOTER last_known_addr { host: "127.12.138.254" port: 44727 } } peers { permanent_uuid: "9cc09b2071ea409587a0ceb8a1d7c94a" member_type: VOTER last_known_addr { host: "127.12.138.253" port: 43519 } } peers { permanent_uuid: "3c53c36ddfd84d68846da9a01be74fc2" member_type: VOTER last_known_addr { host: "127.12.138.252" port: 44637 } } }
I20260811 02:56:47.133379 7019 sys_catalog.cc:458] T 00000000000000000000000000000000 P 71018b8f50c84cf4991af4f7257992a4 [sys.catalog]: This master's current role is: LEADER
I20260811 02:56:47.140014 7021 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 02:56:47.143731 7021 catalog_manager.cc:1504] Loading token signing keys...
I20260811 02:56:47.157743 7021 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 71018b8f50c84cf4991af4f7257992a4: Generated new TSK 0
I20260811 02:56:47.158720 7021 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260811 02:56:47.186690 6982 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 02:56:47.182780 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35338 (local address 127.12.138.254:44727)
0811 02:56:47.182926 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0811 02:56:47.182931 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 02:56:47.185094 (+ 2163us) server_negotiation.cc:407] Connection header received
0811 02:56:47.185606 (+ 512us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:56:47.185611 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:56:47.185685 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:56:47.185748 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:56:47.186515 (+ 767us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:35338: BlockingRecv error: recv got EOF from 127.0.0.1:35338 (error 108)
Metrics: {"server-negotiator.queue_time_us":56}
W20260811 02:56:47.188498 6985 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 02:56:47.183622 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56590 (local address 127.12.138.253:43519)
0811 02:56:47.183766 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0811 02:56:47.183771 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 02:56:47.185506 (+ 1735us) server_negotiation.cc:407] Connection header received
0811 02:56:47.187582 (+ 2076us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:56:47.187587 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:56:47.187657 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:56:47.187722 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:56:47.188359 (+ 637us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:56590: BlockingRecv error: recv got EOF from 127.0.0.1:56590 (error 108)
Metrics: {"server-negotiator.queue_time_us":67}
W20260811 02:56:47.190017 6981 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 02:56:47.184325 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50442 (local address 127.12.138.252:44637)
0811 02:56:47.184451 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0811 02:56:47.184456 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 02:56:47.187321 (+ 2865us) server_negotiation.cc:407] Connection header received
0811 02:56:47.189114 (+ 1793us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:56:47.189118 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:56:47.189218 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:56:47.189279 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:56:47.189889 (+ 610us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:50442: BlockingRecv error: recv got EOF from 127.0.0.1:50442 (error 108)
Metrics: {"server-negotiator.queue_time_us":49}
W20260811 02:56:47.195953 6985 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 02:56:47.194007 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56600 (local address 127.12.138.253:43519)
0811 02:56:47.194132 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0811 02:56:47.194137 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 02:56:47.194698 (+ 561us) server_negotiation.cc:407] Connection header received
0811 02:56:47.194993 (+ 295us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:56:47.194997 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:56:47.195090 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:56:47.195151 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:56:47.195825 (+ 674us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:56600: BlockingRecv error: recv got EOF from 127.0.0.1:56600 (error 108)
Metrics: {"server-negotiator.queue_time_us":48}
W20260811 02:56:47.202040 6985 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 02:56:47.200054 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56610 (local address 127.12.138.253:43519)
0811 02:56:47.200182 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0811 02:56:47.200187 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 02:56:47.200801 (+ 614us) server_negotiation.cc:407] Connection header received
0811 02:56:47.201010 (+ 209us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:56:47.201013 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:56:47.201068 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:56:47.201126 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:56:47.201917 (+ 791us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:56610: BlockingRecv error: recv got EOF from 127.0.0.1:56610 (error 108)
Metrics: {"server-negotiator.queue_time_us":46}
W20260811 02:56:47.207876 6985 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 02:56:47.206318 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56612 (local address 127.12.138.253:43519)
0811 02:56:47.206423 (+ 105us) server_negotiation.cc:206] Beginning negotiation
0811 02:56:47.206428 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 02:56:47.206954 (+ 526us) server_negotiation.cc:407] Connection header received
0811 02:56:47.207147 (+ 193us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:56:47.207150 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:56:47.207202 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:56:47.207267 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:56:47.207768 (+ 501us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:56612: BlockingRecv error: recv got EOF from 127.0.0.1:56612 (error 108)
Metrics: {"server-negotiator.queue_time_us":42}
W20260811 02:56:47.215741 6985 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 02:56:47.214080 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56628 (local address 127.12.138.253:43519)
0811 02:56:47.214241 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0811 02:56:47.214245 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 02:56:47.214773 (+ 528us) server_negotiation.cc:407] Connection header received
0811 02:56:47.214966 (+ 193us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:56:47.214969 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:56:47.215020 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:56:47.215073 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:56:47.215629 (+ 556us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:56628: BlockingRecv error: recv got EOF from 127.0.0.1:56628 (error 108)
Metrics: {"server-negotiator.queue_time_us":75}
W20260811 02:56:47.228066 6985 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 02:56:47.226362 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56632 (local address 127.12.138.253:43519)
0811 02:56:47.226477 (+ 115us) server_negotiation.cc:206] Beginning negotiation
0811 02:56:47.226482 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 02:56:47.227046 (+ 564us) server_negotiation.cc:407] Connection header received
0811 02:56:47.227262 (+ 216us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:56:47.227265 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:56:47.227318 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:56:47.227372 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:56:47.227944 (+ 572us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:56632: BlockingRecv error: recv got EOF from 127.0.0.1:56632 (error 108)
Metrics: {"server-negotiator.queue_time_us":45}
W20260811 02:56:47.248795 6985 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 02:56:47.246904 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56644 (local address 127.12.138.253:43519)
0811 02:56:47.247035 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0811 02:56:47.247040 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 02:56:47.247690 (+ 650us) server_negotiation.cc:407] Connection header received
0811 02:56:47.247916 (+ 226us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:56:47.247921 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:56:47.247983 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:56:47.248039 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:56:47.248650 (+ 611us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:56644: BlockingRecv error: recv got EOF from 127.0.0.1:56644 (error 108)
Metrics: {"server-negotiator.queue_time_us":52}
W20260811 02:56:47.286065 6985 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 02:56:47.283881 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56656 (local address 127.12.138.253:43519)
0811 02:56:47.284026 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0811 02:56:47.284031 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 02:56:47.284824 (+ 793us) server_negotiation.cc:407] Connection header received
0811 02:56:47.285097 (+ 273us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:56:47.285101 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:56:47.285208 (+ 107us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:56:47.285278 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:56:47.285931 (+ 653us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:56656: BlockingRecv error: recv got EOF from 127.0.0.1:56656 (error 108)
Metrics: {"server-negotiator.queue_time_us":59}
W20260811 02:56:47.354697 6985 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 02:56:47.353132 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56666 (local address 127.12.138.253:43519)
0811 02:56:47.353334 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0811 02:56:47.353339 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 02:56:47.353540 (+ 201us) server_negotiation.cc:407] Connection header received
0811 02:56:47.353675 (+ 135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:56:47.353679 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:56:47.353746 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:56:47.353863 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:56:47.354554 (+ 691us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:56666: BlockingRecv error: recv got EOF from 127.0.0.1:56666 (error 108)
Metrics: {"server-negotiator.queue_time_us":89}
W20260811 02:56:47.486645 6985 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 02:56:47.485349 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56676 (local address 127.12.138.253:43519)
0811 02:56:47.485501 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0811 02:56:47.485505 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 02:56:47.485824 (+ 319us) server_negotiation.cc:407] Connection header received
0811 02:56:47.485903 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:56:47.485906 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:56:47.485963 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:56:47.486059 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:56:47.486533 (+ 474us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:56676: BlockingRecv error: recv got EOF from 127.0.0.1:56676 (error 108)
Metrics: {"server-negotiator.queue_time_us":57}
W20260811 02:56:47.747272 6985 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 02:56:47.745480 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56688 (local address 127.12.138.253:43519)
0811 02:56:47.745670 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0811 02:56:47.745675 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 02:56:47.745918 (+ 243us) server_negotiation.cc:407] Connection header received
0811 02:56:47.746088 (+ 170us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:56:47.746091 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:56:47.746160 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:56:47.746255 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:56:47.747122 (+ 867us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:56688: BlockingRecv error: recv got EOF from 127.0.0.1:56688 (error 108)
Metrics: {"server-negotiator.queue_time_us":61}
I20260811 02:56:48.070905 7017 catalog_manager.cc:1259] Loaded cluster ID: 49ca592dcc1243b7814e881decd6bea6
I20260811 02:56:48.070997 7017 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 9cc09b2071ea409587a0ceb8a1d7c94a: loading cluster ID for follower catalog manager: success
I20260811 02:56:48.072904 7017 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 9cc09b2071ea409587a0ceb8a1d7c94a: acquiring CA information for follower catalog manager: success
I20260811 02:56:48.074332 7017 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 9cc09b2071ea409587a0ceb8a1d7c94a: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260811 02:56:48.133924 7054 catalog_manager.cc:1259] Loaded cluster ID: 49ca592dcc1243b7814e881decd6bea6
I20260811 02:56:48.134052 7054 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 3c53c36ddfd84d68846da9a01be74fc2: loading cluster ID for follower catalog manager: success
I20260811 02:56:48.136221 7054 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 3c53c36ddfd84d68846da9a01be74fc2: acquiring CA information for follower catalog manager: success
I20260811 02:56:48.137733 7054 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 3c53c36ddfd84d68846da9a01be74fc2: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260811 02:56:48.265988 7138 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:56:48.261500 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56696 (local address 127.12.138.253:43519)
0811 02:56:48.261775 (+ 275us) server_negotiation.cc:206] Beginning negotiation
0811 02:56:48.261780 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 02:56:48.261915 (+ 135us) server_negotiation.cc:407] Connection header received
0811 02:56:48.262091 (+ 176us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:56:48.262094 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:56:48.262258 (+ 164us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:56:48.262405 (+ 147us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 02:56:48.262701 (+ 296us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:56:48.263291 (+ 590us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:56:48.264237 (+ 946us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:56:48.264487 (+ 250us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:56:48.264620 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:56:48.264954 (+ 334us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 02:56:48.265549 (+ 595us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 02:56:48.265715 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0811 02:56:48.265760 (+ 45us) server_negotiation.cc:299] Negotiation successful
0811 02:56:48.265821 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":187,"thread_start_us":70,"threads_started":1}
I20260811 02:56:48.273605 7144 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:56:48.269603 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50444 (local address 127.12.138.252:44637)
0811 02:56:48.269952 (+ 349us) server_negotiation.cc:206] Beginning negotiation
0811 02:56:48.269956 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 02:56:48.269969 (+ 13us) server_negotiation.cc:407] Connection header received
0811 02:56:48.270053 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:56:48.270056 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:56:48.270186 (+ 130us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:56:48.270288 (+ 102us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 02:56:48.270501 (+ 213us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:56:48.271028 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:56:48.271876 (+ 848us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:56:48.272114 (+ 238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:56:48.272230 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:56:48.272585 (+ 355us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 02:56:48.273067 (+ 482us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 02:56:48.273262 (+ 195us) server_negotiation.cc:1035] Waiting for connection context
0811 02:56:48.273322 (+ 60us) server_negotiation.cc:299] Negotiation successful
0811 02:56:48.273378 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":274,"thread_start_us":198,"threads_started":1}
I20260811 02:56:48.282230 7138 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:56:48.277914 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56704 (local address 127.12.138.253:43519)
0811 02:56:48.278078 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0811 02:56:48.278082 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 02:56:48.278192 (+ 110us) server_negotiation.cc:407] Connection header received
0811 02:56:48.278332 (+ 140us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:56:48.278335 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:56:48.278384 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:56:48.278472 (+ 88us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 02:56:48.278906 (+ 434us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:56:48.279747 (+ 841us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:56:48.280884 (+ 1137us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:56:48.281313 (+ 429us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:56:48.281452 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:56:48.281705 (+ 253us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 02:56:48.281804 (+ 99us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 02:56:48.281890 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0811 02:56:48.282001 (+ 111us) server_negotiation.cc:299] Negotiation successful
0811 02:56:48.282070 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260811 02:56:48.282622 7144 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:56:48.278289 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50448 (local address 127.12.138.252:44637)
0811 02:56:48.278413 (+ 124us) server_negotiation.cc:206] Beginning negotiation
0811 02:56:48.278417 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 02:56:48.278899 (+ 482us) server_negotiation.cc:407] Connection header received
0811 02:56:48.279075 (+ 176us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:56:48.279078 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:56:48.279134 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:56:48.279211 (+ 77us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 02:56:48.279505 (+ 294us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:56:48.280238 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:56:48.281352 (+ 1114us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:56:48.281683 (+ 331us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:56:48.281780 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:56:48.282089 (+ 309us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 02:56:48.282172 (+ 83us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 02:56:48.282226 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0811 02:56:48.282429 (+ 203us) server_negotiation.cc:299] Negotiation successful
0811 02:56:48.282496 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260811 02:56:48.282999 7150 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:56:48.277506 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35352 (local address 127.12.138.254:44727)
0811 02:56:48.277807 (+ 301us) server_negotiation.cc:206] Beginning negotiation
0811 02:56:48.277815 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 02:56:48.277880 (+ 65us) server_negotiation.cc:407] Connection header received
0811 02:56:48.278171 (+ 291us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:56:48.278175 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:56:48.278378 (+ 203us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:56:48.278480 (+ 102us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 02:56:48.278989 (+ 509us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:56:48.279781 (+ 792us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:56:48.280957 (+ 1176us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:56:48.281371 (+ 414us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:56:48.281487 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:56:48.282046 (+ 559us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 02:56:48.282461 (+ 415us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 02:56:48.282540 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0811 02:56:48.282779 (+ 239us) server_negotiation.cc:299] Negotiation successful
0811 02:56:48.282853 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":219,"thread_start_us":81,"threads_started":1}
I20260811 02:56:48.287298 12843 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDawEwS/build/debug/bin/kudu with pid 6713
I20260811 02:56:48.302345 12843 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDawEwS/build/debug/bin/kudu with pid 6804
I20260811 02:56:48.312287 12843 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDawEwS/build/debug/bin/kudu with pid 6891
2026-08-11T02:56:48Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1 (3341 ms)
[----------] 2 tests from ConnectToFollowerMasterTest (6750 ms total)
[----------] Global test environment tear-down
[==========] 56 tests from 4 test suites ran. (300603 ms total)
[ PASSED ] 55 tests.
[ FAILED ] 1 test, listed below:
[ FAILED ] SecurityITest.TestJwtMiniCluster
1 FAILED TEST